skip to main content


Title: Designing a Virtual Client for Requirements Elicitation Interviews.
[Context and motivation] Role-playing offers experiential learning through the simulation of real-world scenarios; for this reason, it is widely used in software engineering education. In Requirements Engineering, role-playing is a popular way to provide students hands-on experience with requirements elicitation interviews. [Problem] However, managing a role-playing activity to simulate requirements elicitation interviews in a class is time-consuming, as it often requires pairing students with student assistants or fellow classmates who act as either customers or requirements analysts as well as creating and maintaining the interview schedules between the actors. To make the adoption of role-playing activities in a class feasible, there is a need to develop a solution to reduce instructors’ workload. [Principal ideas] To solve this problem we propose the use of VIrtual CustOmer (VICO), an intent-based, multimodal, conversational agent. VICO offers an interview experience comparable to talking to a human and provides a transcript of the interview annotated with the mistakes students made in it. The adoption of VICO will eliminate the need to schedule interviews as the students can interact with it in their free time. Moreover, the transcript of the interview allows students to evaluate their performance to refine and improve their interviewing skills. [Contribution] In this research preview, we show the architecture of VICO and how it can be developed using existing technologies, we provide an online rule-based initial prototype and show the practicality and applicability of this tool through an exploratory study.  more » « less
Award ID(s):
1718377
NSF-PAR ID:
10174184
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Requirements Engineering: Foundation for Software Quality. REFSQ 2020
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Interviews are the most widely used elicitation technique in requirements engineering (RE). However, conducting a requirements elicitation interview is challenging. The mistakes made in design or conduct of the interviews can create problems in the later stages of requirements analysis. Empirical evidence about effective pedagogical approaches for training novices on conducting requirements elicitation interviews is scarce. In this paper, we present a novel pedagogical approach for training student analysts in the art of elicitation interviews. Our study is conducted in two parts: first, we perform an observational study of interviews performed by novices, and we present a classification of the most common mistakes made; second, we utilize this list of mistakes and monitor the students’ progress in three set of interviews to discover the individual areas for improvement. We conducted an empirical study involving role-playing and authentic assessment in two semesters on two different cohorts of students. In the first semester, we had 110 students, teamed up in 28 groups, to conduct three interviews with stakeholders. We qualitatively analysed the data to identify and classify the mistakes made from their first interview only. In the second semester, we had 138 students in 34 groups and we monitored and analysed their progress in all three interviews by utilizing the list of mistakes from the first study. First, we identified 34 unique mistakes classified into seven high-level themes, namely question formulation, question omission, interview order, communication skills, analyst behaviour, customer interaction, teamwork and planning. In the second study, we discovered that the students struggled mostly in the areas of question formulation, question omission and interview order and did not manage to improve their skills throughout the three interviews. Our study presents a novel and repeatable pedagogical design, and our findings extend the body of knowledge aimed at RE education and training by providing an empirically grounded categorization of mistakes made by novices. We offer an analysis of the main pain points in which instructors should pay more attention during their design and training. 
    more » « less
  2. Purpose The goal of this study is to explore an immediate step in understanding the lived experiences of under-represented students through metaphor construction and possibly collect more in-depth data through photograph-based interviews. Design/Methodology/Approach This article introduced photo-elicitation based narrative interviews as a qualitative methodology while interviewing fourteen undergraduate community college students mostly from underrepresented groups (URGs). At the beginning of each interview, the authors probed the participants with 8 photographs chosen by the research team to represent a diverse set of experiences in engineering. The authors conducted a thematic analysis of the interview data. Findings The findings suggested that the inclusion of photo-elicitation often catalyzed consumption of representations, images, metaphors, and voice to stories passed unnoticed; and finally produces more detailed descriptions and complements semi-structured narrative interviews. Research Limitations/Implications This study advances the scholarship that extends photograph driven interviews/photo elicitation methodology while interviewing marginalized population and offers a roadmap for what a multi-modal, arts-based analysis process might look like for in-depth interviews. Practical Implications The use of photo-elicitation in our research enabled a deeper, more poignant exploration of the URG students' experience of navigating engineering. The participants were able to relate to the photographs and shared their life narratives through them; hence, use of photographs can be adapted in future research. Social Implications Our research revealed that PEI has excellent potential to capture marginalized narratives of URGs, which is not well explored in educational research, specially, in higher education. In our research, PEI promoted more culturally inclusive approaches positioning the participants as experts of their own narratives. Originality/Value The study presented in this paper serves as an example of qualitative research that expands methodological boundaries and centers the role of power, marginalization, and creativity in research. This work serves as a unique and important contribution to the photo-elicitation literature, offering a critical roadmap for researchers who are drawn to photo elicitation/photograph driven interviews as a method to explore their inquiry. 
    more » « less
  3. Requirements are elicited from the customer and other stakeholders through an iterative process of interviews, prototyping, and other interactive sessions. Then, requirements can be further extended, based on the analysis of the features of competing products available on the market. Understanding how this process takes place can help to identify the contribution of the different elicitation phases, thereby allowing requirements analysts to better distribute their resources. In this work, we empirically study in which way requirements get transformed from initial ideas into documented needs, and then evolve based on the inspiration coming from similar products. To this end, we select 30 subjects that act as requirements analysts, and we perform interview-based elicitation sessions with a fictional customer. After the sessions, the analysts produce a first set of requirements for the system. Then, they are required to search similar products in the app stores and extend the requirements, inspired by the identified apps. The requirements documented at each step are evaluated, to assess to which extent and in which way the initial idea evolved throughout the process. Our results show that only between 30% and 38% of the requirements produced after the interviews include content that can be fully traced to initial customer’s ideas. The rest of the content is dedicated to new requirements, and up to 21% of it belongs to completely novel topics. Furthermore, up to 42% of the requirements inspired by the app stores cover additional features compared to the ones identified after the interviews. The results empirically show that requirements are not elicited in strict sense, but actually co-created through interviews, with analysts playing a crucial role in the process. In addition, we show evidence that app store-inspired elicitation can be particularly beneficial to complete the requirements. 
    more » « less
  4. [Context] Interviews are the most widely used elicitation technique in requirements engineering. However, conducting effective requirements elicitation interviews is challenging, due to the combination of technical and soft skills that requirements analysts often acquire after a long period of professional practice. Empirical evidence about training the novices on conducting effective requirements elicitation interviews is scarce. [Objectives] We present a list of most common mistakes that novices make in requirements elicitation interviews. The objective is to assist the educators in teaching interviewing skills to student analysts. [Research Method] We conducted an empirical study involving role-playing and authentic assessment with 110 students, teamed up in 28 groups, to conduct interviews with a customer. One researcher made observation notes during the interview while two researchers reviewed the recordings. We qualitatively analyzed the data to identify the themes and classify the mistakes. [Results and conclusion] We identified 34 unique mistakes classified into 7 high level themes. We also give examples of the mistakes made by the novices in each theme, to assist the educationists and trainers. Our research design is a novel combination of well-known pedagogical approaches described in sufficient details to make it repeatable for future requirements engineering education and training research. 
    more » « less
  5. In higher education, our students experience a wide range of vulnerabilities, which we define as a lack of physical, social, and emotional security. Vulnerabilities are unevenly distributed and stratified by race, gender, and socioeconomic status. What is the role of vulnerability in facilitating the development and expansion of capabilities, a core mission of higher education in many Western nations? On the one hand, a lack of resources can substantially undermine students’ abilities to learn and integrate new knowledge. On the other hand, vulnerability has been theorized as a catalyst for transformation, a condition of suffering and fragility that engenders change. Operational definitions of vulnerability in higher education need to acknowledge its dual-sided nature and potential to help and harm student growth. In this paper we ask what kinds of vulnerability facilitate and inhibit students’ development of capabilities? To guide our thinking, we analyze the life history interviews of three engineering students attending a liberal arts college in the Northeastern United States: one American student of above-average academic performance (representing the normative case), one immigrant student of color of above-average academic performance, and one immigrant student of color of below-average academic performance. Utilizing qualitative structured coding methods, we coded each interview using Walker’s (2006) capabilities list for higher education contexts. We also inductively coded instances of vulnerability that arose during the interviews, which often overlapped with one or more of Walker’s capabilities, and noted their proximity to other capabilities at that time in their lives. Coding was performed by three members of the research team using consensus coding techniques to reduce individual biases. We suggest that vulnerability acts as a conversion factor, which both enables and inhibits capability development. Vulnerability is often the product of structural factors, which distribute vulnerability unequally by gender, race, social class, and country of origin. However, the valence of vulnerability is mediated by individual agency, through which individuals may experience transformation through reframing vulnerability as personal triumph over adversity. We argue that the capabilities approach offers a better balance between structure and agency than two competing models, shame resilience theory and psychological safety. This study contributes to new ways of conceptualizing and measuring vulnerability and human development at the micro-level in universities. Higher education systems are central to citizens’ capability development, and understanding student vulnerabilities helps such systems respond to rapid societal changes. 
    more » « less