More Like this

1. Learning from Mistakes: An Empirical Study of Elicitation Interviews Performed by Novices

   https://doi.org/10.1109/RE.2018.00027  

   Bano, Muneera; Bano, Muneera; Zowghi, Didar; Ferrari, Alessio; Spoletini, Paola; Donati, Beatrice (August 2018, 2018 IEEE 26th International Requirements Engineering Conference (RE))

   [Context] Interviews are the most widely used elicitation technique in requirements engineering. However, conducting effective requirements elicitation interviews is challenging, due to the combination of technical and soft skills that requirements analysts often acquire after a long period of professional practice. Empirical evidence about training the novices on conducting effective requirements elicitation interviews is scarce. [Objectives] We present a list of most common mistakes that novices make in requirements elicitation interviews. The objective is to assist the educators in teaching interviewing skills to student analysts. [Research Method] We conducted an empirical study involving role-playing and authentic assessment with 110 students, teamed up in 28 groups, to conduct interviews with a customer. One researcher made observation notes during the interview while two researchers reviewed the recordings. We qualitatively analyzed the data to identify the themes and classify the mistakes. [Results and conclusion] We identified 34 unique mistakes classified into 7 high level themes. We also give examples of the mistakes made by the novices in each theme, to assist the educationists and trainers. Our research design is a novel combination of well-known pedagogical approaches described in sufficient details to make it repeatable for future requirements engineering education and training research. 

   more » « less
2. Designing a Virtual Client for Requirements Elicitation Interviews.

   https://doi.org/10.1007/978-3-030-44429-7_12  

   Debnath, S.; Spoletini, P. (January 2020, Requirements Engineering: Foundation for Software Quality. REFSQ 2020)

   [Context and motivation] Role-playing offers experiential learning through the simulation of real-world scenarios; for this reason, it is widely used in software engineering education. In Requirements Engineering, role-playing is a popular way to provide students hands-on experience with requirements elicitation interviews. [Problem] However, managing a role-playing activity to simulate requirements elicitation interviews in a class is time-consuming, as it often requires pairing students with student assistants or fellow classmates who act as either customers or requirements analysts as well as creating and maintaining the interview schedules between the actors. To make the adoption of role-playing activities in a class feasible, there is a need to develop a solution to reduce instructors’ workload. [Principal ideas] To solve this problem we propose the use of VIrtual CustOmer (VICO), an intent-based, multimodal, conversational agent. VICO offers an interview experience comparable to talking to a human and provides a transcript of the interview annotated with the mistakes students made in it. The adoption of VICO will eliminate the need to schedule interviews as the students can interact with it in their free time. Moreover, the transcript of the interview allows students to evaluate their performance to refine and improve their interviewing skills. [Contribution] In this research preview, we show the architecture of VICO and how it can be developed using existing technologies, we provide an online rule-based initial prototype and show the practicality and applicability of this tool through an exploratory study. 

   more » « less
3. Teaching requirements elicitation interviews: an empirical study of learning from mistakes

   https://doi.org/10.1007/s00766-019-00313-0  

   Bano, Muneera; Zowghi, Didar; Ferrari, Alessio; Spoletini, Paola; Donati, Beatrice (January 2019, Requirements Engineering)

   Interviews are the most widely used elicitation technique in requirements engineering (RE). However, conducting a requirements elicitation interview is challenging. The mistakes made in design or conduct of the interviews can create problems in the later stages of requirements analysis. Empirical evidence about effective pedagogical approaches for training novices on conducting requirements elicitation interviews is scarce. In this paper, we present a novel pedagogical approach for training student analysts in the art of elicitation interviews. Our study is conducted in two parts: first, we perform an observational study of interviews performed by novices, and we present a classification of the most common mistakes made; second, we utilize this list of mistakes and monitor the students’ progress in three set of interviews to discover the individual areas for improvement. We conducted an empirical study involving role-playing and authentic assessment in two semesters on two different cohorts of students. In the first semester, we had 110 students, teamed up in 28 groups, to conduct three interviews with stakeholders. We qualitatively analysed the data to identify and classify the mistakes made from their first interview only. In the second semester, we had 138 students in 34 groups and we monitored and analysed their progress in all three interviews by utilizing the list of mistakes from the first study. First, we identified 34 unique mistakes classified into seven high-level themes, namely question formulation, question omission, interview order, communication skills, analyst behaviour, customer interaction, teamwork and planning. In the second study, we discovered that the students struggled mostly in the areas of question formulation, question omission and interview order and did not manage to improve their skills throughout the three interviews. Our study presents a novel and repeatable pedagogical design, and our findings extend the body of knowledge aimed at RE education and training by providing an empirically grounded categorization of mistakes made by novices. We offer an analysis of the main pain points in which instructors should pay more attention during their design and training. 

   more » « less
4. Towards Establishing Consistent Proposal Binning Methods for Unimodal and Multimodal Interaction Elicitation Studies

   https://doi.org/10.1007/978-3-031-05311-5  

   Xiaoyan Zhou, Adam S. (June 2022, HCII: International Conference on Human-Computer Interaction)

   The Open University of Japan, Chiba (Ed.)

   More than two hundred papers on elicitation studies have been published in the last ten years. These works are mainly focused on generating user-defined gesture sets and discovering natural feeling multimodal interaction techniques with virtual objects. Few papers have discussed binning the elicited interaction proposals after data collection. Binning is a process of grouping the entire set of user-generated interaction proposals based on similarity criteria. The binned set of proposals is then analyzed to produce a consensus set, which results in the user-defined interaction set. This paper presents a formula to use when deciding how to bin interaction proposals, thus helping to establish a more consistent binning procedure. This work can provide human-computer interaction (HCI) researchers with the guidance they need for interaction elicitation data processing, which is largely missing from current elicitation study literature. Using this approach will improve the efficiency and effectiveness of the binning process, increase the reliability of us er-defined interaction sets, and most importantly, improve the replicability of elicitation studies. 

   more » « less
5. The Impacts of Referent Display on Gesture and Speech Elicitation

   https://doi.org/10.1109/TVCG.2022.3203090  

   Williams, Adam S.; Ortega, Francisco R. (July 2022, IEEE Transactions on Visualization and Computer Graphics)

   Elicitation studies have become a popular method of participatory design. While traditionally used to examine unimodal gesture interactions, elicitation has started being used with other novel interaction modalities. Unfortunately, there has been no work that examines the impact of referent display on elicited interaction proposals. To address that concern this work provides a detailed comparison between two elicitation studies that were similar in design apart from the way that participants were prompted for interaction proposals (i.e., the referents). Based on this comparison the impact of referent display on speech and gesture interaction proposals are each discussed. The interaction proposals between these elicitation studies were not identical. Gesture proposals were the least impacted by referent display, showing high proposal similarity between the two works. Speech proposals were highly biased by text referents with proposals directly mirroring text-based referents an average of 69.36% of the time. In short, the way that referents are presented during elicitation studies can impact the resulting interaction proposals; however, the level of impact found is dependent on the modality of input elicited. 

   more » « less