An official website of the United States government
This theory paper focuses on a research methodology, using an autoethnographic approach to reflect on the use of cognitive interviewing (CI) as a method of increasing the quality and validity of questionnaires in pre-validation design and development stages. We first provide a brief review of cognitive interviewing, sometimes called “cognitive think-aloud interviewing” or “think-aloud interviewing,” before presenting a summary of two studies conducted by the authors that used CI. Differences between these two studies are discussed as comparative cases and advice is given to scholars considering the use of CI in their own research. While this paper is not an explicit guide to conducting CI, we do intend to provide advice and wisdom for researchers who are unfamiliar with CI as a method, grounded in our experience with the method. This paper is written with a particular focus on the use of CI in engineering education research (EER) but may be more broadly applicable to other social sciences domains.
more »
« less
This content will become publicly available on October 13, 2025
Special Session: Improving Interview Skills Using the Interview Quality Reflection Tool (IQRT)
Engineering education research often uses interview-based studies to develop critical theoretical findings for transforming engineering education and practice. However, researchers are often left with few practical tools to hone the craft of interviewing, especially for unscripted interactions or moments that go beyond predefined protocols. Here, we introduce a process-oriented tool designed to refine researchers' interviewing capabilities. By focusing on adaptability and reflexivity, this tool aims to elevate the quality of evaluative discussions between novice researchers and their mentors.
more »
« less
- Award ID(s):
- 2045392
- PAR ID:
- 10586674
- Publisher / Repository:
- IEEE
- Date Published:
- ISBN:
- 979-8-3503-5150-7
- Page Range / eLocation ID:
- 1 to 3
- Format(s):
- Medium: X
- Location:
- Washington, DC, USA
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This methods paper presents the interview quality reflection tool (IQRT) to evaluate the quality of qualitative research interviews. Qualitative researchers commonly use semi-structured interviews that rely on the interviewers’ ability to improvise in real time based on the needs of the study. Given that interviewing involves numerous tacit skills that cannot be delineated by a simple written protocol, it is necessary that researchers develop interview competencies through practice and reflection. While prior literature on interviewing has often focused on developing interview protocols, we know little about how interviewers themselves may be trained to gather high-quality data. In this paper, we focus on how the IQRT may be used to guide the self-assessment of research interviews. We discuss how interviews are used in engineering education, how we developed and applied the IQRT, and how lessons learned through using this tool might lead to improved interviewing skills through careful examination of interview structure, content, and context within the mentoring process.more » « less
-
Engineering education research heavily relies on qualitative studies that utilize interview-based approaches. The quality and depth of knowledge derived from these studies depend heavily on the craft of conducting interviews, a facet often overlooked in prior work on qualitative methods. This special session aims to address this gap by guiding engineering education researchers in honing their interviewing skills for qualitative research. Participants will learn best practices for developing interview protocols, creating an accessible environment, and capturing high-quality data. Through case studies and hands-on activities, attendees will gain confidence in moderating conversations, improving data collection, and enhancing their overall skillset. This session provides an opportunity for researchers interested in qualitative research and scholarly educators to deepen their understanding of conducting meaningful interviews. By bridging the gap between the importance of qualitative studies and the need for skilled interviewers, we aim to contribute to the advancement of engineering education research.more » « less
-
Employment outcomes for autistic 1 individuals are often poorer relative to their neurotypical (NT) peers, resulting in a greater need for other forms of financial and social support. While a great deal of work has focused on developing interventions for autistic children, relatively less attention has been paid to directly addressing the employment challenges faced by autistic adults. One key impediment to autistic individuals securing employment is the job interview. Autistic individuals often experience anxiety in interview situations, particularly with open-ended questions and unexpected interruptions. They also exhibit atypical gaze patterns that may be perceived as, but not necessarily indicative of, disinterest or inattention. In response, we developed a closed-loop adaptive virtual reality (VR)–based job interview training platform, which we have named Career Interview Readiness in VR (CIRVR). CIRVR is designed to provide an engaging, adaptive, and individualized experience to practice and refine interviewing skills in a less anxiety-inducing virtual context. CIRVR contains a real-time physiology-based stress detection module, as well as a real-time gaze detection module, to permit individualized adaptation. We also present the first prototype of the CIRVR Dashboard, which provides visualizations of data to help autistic individuals as well as potential employers and job coaches make sense of the data gathered from interview sessions. We conducted a feasibility study with 9 autistic and 8 NT individuals to assess the preliminary usability and feasibility of CIRVR. Results showed differences in perceived usability of the system between autistic and NT participants, and higher levels of stress in autistic individuals during interviews. Participants across both groups reported satisfaction with CIRVR and the structure of the interview. These findings and feedback will support future work in improving CIRVR’s features in hopes for it to be a valuable tool to support autistic job candidates as well as their potential employers.more » « less
