An official website of the United States government
This Work-in-Progress Research paper describes (1) the contemporary research space on ethics education in engineering; (2) our long-term research plan; (3) the theoretical underpinnings of Phase 1 of our research plan (phenomenography); and (4) the design and developmental process of a phenomenographic interview protocol to explore engineers’ experiences with ethics. Ethical behavior is a complex phenomenon that is complicated by the institutional and cultural contexts in which it occurs. Engineers also have varied roles and often work in a myriad of capacities that influence their experiences with and understanding of ethics in practice. We are using phenomenography, a qualitative research approach, to explore and categorize the ways engineers experience and understand ethical engineering practice. Specifically, phenomenography will allow us to systematically investigate the range and complexity of ways that engineers experience ethics in professional practice in the health products industry. Phenomenographic data will be obtained through a specialized type of semi-structured interview. Here we introduce the design of our interview protocol and its four sections: Background, Experience, Conceptual, and Summative. We also describe our iterative process for framing questions throughout each section.
more »
« less
Work In Progress: Considering the Impact on Research Quality of a Team Approach to Phenomenography
This WIP paper describes a team approach to phenomenography on ethical engineering practice in the health products industry and its potential impact on research quality. Although qualitative researchers often conduct phenomenography collaboratively, most often a single individual leads the data collection and analysis; others primarily serve as critical reviewers. However, quality may be enhanced by involving collaborators as data analysts in “sustained cycles of scrutiny, debate and testing against the data” [1, p. 88], thus interweaving unique perspectives and insights throughout the analysis process. Nonetheless, collaborating in this intensive data analysis process also presents unique challenges. In this paper, we (1) describe the processes we are applying in an integrated team-based phenomenographic study, (2) identify how the team approach affects research quality, and (3) reflect on the challenges inherent to this process. We ground this reflective case study in the methodological literature on phenomenography. Our team strategies include multiple interviewers (and, when possible, two interviewers per inter-view), team communication through reflective memos, and integration of individual and team-based data analysis with peer critique of individual analyses. We compare our team approach with typical individual phenomenographic approaches, and we align our procedures with the five strategies of the Qualifying Qualitative Research Quality Framework, or Q3, designed by Walther, Sochacka, and Kellam [2]. In aligning strategies, we consider benefits and trade-offs.
more »
« less
- Award ID(s):
- 1737303
- PAR ID:
- 10312452
- Date Published:
- Journal Name:
- Work In Progress: Considering the Impact on Research Quality of a Team Approach to Phenomenography
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Purpose Personas are lifelike characters that are driven by potential or real users’ personal goals and experiences when interacting with a product. Personas support user-centered design by focusing on real users’ needs. However, the use of personas in educational research and design requires certain adjustments from its original use in human-computer interface design. This paper aims to propose a process of creating personas from phenomenographic studies, which helps us create data-grounded personas effectively. Design/methodology/approach Personas have features that can help address design problems in educational contexts. The authors compare the use of personas with other common methodologies in education research, including phenomenology and phenomenography. Then, this study presents a six-step process of building personas using phenomenographic study as follows: articulate a design problem, collect user data, assemble phenomenographic categories, build personas, check personas and solve the design problem using personas. The authors illustrate this process with two examples, including the redesign of a professional development website and an undergraduate research program design. Findings The authors find that personas are valuable tools for educational design websites and programs. Phenomenography can productively help educational designers and researchers build sets of personas following the process the authors propose. Originality/value The use and method of personas in educational contexts are scarce and vague. Using the example contexts, the authors provide educational designers and researchers a clear method of creating personas that are relatable and applicable for their design problems.more » « less
-
This methods paper presents the interview quality reflection tool (IQRT) to evaluate the quality of qualitative research interviews. Qualitative researchers commonly use semi-structured interviews that rely on the interviewers’ ability to improvise in real time based on the needs of the study. Given that interviewing involves numerous tacit skills that cannot be delineated by a simple written protocol, it is necessary that researchers develop interview competencies through practice and reflection. While prior literature on interviewing has often focused on developing interview protocols, we know little about how interviewers themselves may be trained to gather high-quality data. In this paper, we focus on how the IQRT may be used to guide the self-assessment of research interviews. We discuss how interviews are used in engineering education, how we developed and applied the IQRT, and how lessons learned through using this tool might lead to improved interviewing skills through careful examination of interview structure, content, and context within the mentoring process.more » « less
-
null (Ed.)In this paper, we share our approach and the process for qualitative analysis of online video data recorded during an after-school robotics program that emphasized computational thinking (CT). Online research strategies may be necessary for various reasons such as when working with a geographically distributed research team, when conducting research with students in an online program, or when resources are inaccessible due to campus closures like those experienced during the COVID-19 pandemic. We followed a three-stage process during qualitative analysis of the videos that included planning and setup, online analysis of videos, and structural coding of memos to explore patterns across the data. Analysis was conducted with a combination of technologies including Google Drive for collaborative coding online and NVivo to collate and summarize findings. The methods and process we describe are readily applicable to other research studies that include video as part of the data set.more » « less
-
Abstract Interviewers’ postinterview evaluations of respondents’ performance (IEPs) are paradata, used to describe the quality of the data obtained from respondents. IEPs are driven by a combination of factors, including respondents’ and interviewers’ sociodemographic characteristics and what actually transpires during the interview. However, relatively few studies examine how IEPs are associated with features of the response process, including facets of the interviewer-respondent interaction and patterns of responding that index data quality. We examine whether features of the response process—various respondents’ behaviors and response quality indicators—are associated with IEPs in a survey with a diverse set of respondents focused on barriers and facilitators to participating in medical research. We also examine whether there are differences in IEPs across respondents’ and interviewers’ sociodemographic characteristics. Our results show that both respondents’ behaviors and response quality indicators predict IEPs, indicating that IEPs reflect what transpires in the interview. In addition, interviewers appear to approach the task of evaluating respondents with differing frameworks, as evidenced by the variation in IEPs attributable to interviewers and associations between IEPs and interviewers’ gender. Further, IEPs were associated with respondents’ education and ethnoracial identity, net of respondents’ behaviors, response quality indicators, and sociodemographic characteristics of respondents and interviewers. Future research should continue to build on studies that examine the correlates of IEPs to better inform whether, when, and how to use IEPs as paradata about the quality of the data obtained.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.18260/1-2--33601
