This Research paper discusses the opportunities that utilizing a computer program can present in analyzing large amounts of qualitative data collected through a survey tool. When working with longitudinal qualitative data, there are many challenges that researchers face. The coding scheme may evolve over time requiring re-coding of early data. There may be long periods of time between data analysis. Typically, multiple researchers will participate in the coding, but this may introduce bias or inconsistencies. Ideally the same researchers would be analyzing the data, but often there is some turnover in the team, particularly when students assist with the coding. Computer programs can enable automated or semi-automated coding helping to reduce errors and inconsistencies in the coded data. In this study, a modeling survey was developed to assess student awareness of model types and administered in four first-year engineering courses across the three universities over the span of three years. The data collected from this survey consists of over 4,000 students’ open-ended responses to three questions about types of models in science, technology, engineering, and mathematics (STEM) fields. A coding scheme was developed to identify and categorize model types in student responses. Over two years, two undergraduate researchers analyzed a total of 1,829 students’ survey responses after ensuring intercoder reliability was greater than 80% for each model category. However, with much data remaining to be coded, the research team developed a MATLAB program to automatically implement the coding scheme and identify the types of models students discussed in their responses. MATLAB coded results were compared to human-coded results (n = 1,829) to assess reliability; results matched between 81%-99% for the different model categories. Furthermore, the reliability of the MATLAB coded results are within the range of the interrater reliability measured between the 2 undergraduate researchers (86-100% for the five model categories). With good reliability of the program, all 4,358 survey responses were coded; results showing the number and types of models identified by students are presented in the paper.
more »
« less
Mitigating Ceiling Effects in a Longitudinal Study of Doctoral Engineering Student Stress and Persistence
Aim/Purpose: The research reported here aims to demonstrate a method by which novel applications of qualitative data in quantitative research can resolve ceiling effect tensions for educational and psychological research.Background: Self-report surveys and scales are essential to graduate education and social science research. Ceiling effects reflect the clustering of responses at the highest response categories resulting in non-linearity, a lack of variability which inhibits and distorts statistical analyses. Ceiling effects in stress reported by students can negatively impact the accuracy and utility of the resulting data.Methodology: A longitudinal sample example from graduate engineering students’ stress, open-ended critical events, and their early departure from doctoral study considerations demonstrate the utility and improved accuracy of adjusted stress measures to include open-ended critical event responses. Descriptive statistics are used to describe the ceiling effects in stress data and adjusted stress data. The longitudinal stress ratings were used to predict departure considerations in multilevel modeling ANCOVA analyses and demonstrate improved model predictiveness.Contribution: Combining qualitative data from open-ended responses with quantitative survey responses provides an opportunity to reduce ceiling effects and improve model performance in predicting graduate student persistence. Here, we present a method for adjusting stress scale responses by incorporating coded critical events based on the Taxonomy of Life Events, the application of this method in the analysis of stress responses in a longitudinal data set, and potential applications.Findings: The resulting process more effectively represents the doctoral student experience within statistical analyses. Stress and major life events significantly impact engineering doctoral students’ departure considerations.Recommendations for Practitioners: Graduate educators should be aware of students’ life events and assist students in managing graduate school expectations while maintaining progress toward their degree. Recommendation for Researchers: Integrating coded open-ended qualitative data into statistical models can increase the accuracy and representation of the lived student experience. The new approach improves the accuracy and presentation of students’ lived experiences by incorporating qualitative data into longitudinal analyses. The improvement assists researchers in correcting data with ceiling effects for use in longitudinal analyses.Impact on Society: The method described here provides a framework to systematically include open-ended qualitative data in which ceiling effects are present.Future Research: Future research should validate the coding process in similar samples and in samples of doctoral students in different fields and master’s students.
more »
« less
- Award ID(s):
- 1844878
- NSF-PAR ID:
- 10464169
- Publisher / Repository:
- Informing Science Institute
- Date Published:
- Journal Name:
- IJEE International Journal of Engineering Education
- Volume:
- 39
- Issue:
- 1
- ISSN:
- 2540-9808
- Page Range / eLocation ID:
- 199 to 227
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Purpose The purpose of this study is to examine how student agency influences career decision-making for doctoral students in biological sciences. The authors address the following questions: How do biological science graduate students navigate career indecision? And how does agency relate to their experiences with career indecision? Design/methodology/approach The authors analyzed interview data collected from 84 PhD biology graduate students. Researchers used a grounded theory approach. After open codes were developed and data were coded, code reports were generated, which were used to determine themes. Findings More than half of the sample had not committed to a career path, and undecided students were bifurcated into two categories: Uncommitted and Uncertain. Uncommitted graduate students demonstrated agency in their approach and were focused on exploration and development. Uncertain students demonstrated less agency, were more fearful and perceived less control and clarity about their options and strategies to pursue career goals. Practical implications Findings suggest some forms of indecision can be productive and offer institutional leaders guidance for increasing the efficacy of career development and exploration programming. Originality/value Research on doctoral student career decision-making is often quantitative and rarely explores the role of agency. This qualitative study focuses on the relationship between student agency and career indecision, which is an understudied aspect of career development.more » « less
-
more » « less
Abstract Background Although most engineering graduate students are funded and usually complete their degrees faster than other disciplines, attrition remains a problem in engineering. Existing research has explored the psychological and sociological factors contributing to attrition but not the structural factors impacting attrition.
Purpose/Hypothesis Using systems theory, this study seeks to understand nuance in how underlying structural causes affect engineering graduate students' attrition experiences in ways that may differ from their official reasons for departure.
Design/Methods Data were collected through semi‐structured interviews with seven departing or already departed engineering doctoral students from R1 graduate programs across the United States. Using thematic analysis, root cause analyses were conducted to understand participants' attrition experiences to explore how structures influence causes of departure.
Results The ways participants discuss root causes of their departure indicate differences in formal reasons for departure and underlying causes of departure. We highlight the role of informal and formal policy as root causes of a different attrition rationale often passed off as interpersonal issues. When interpreted as evidence of structural issues, the causes of departure show ways in which action–inaction, policy–“null” policy serve as structural features governing student attrition decision processes. We also highlight a form of benign neglect toward struggling graduate students.
Conclusion This study reveals important nuances underlying face‐value reasons of attrition indicating foundational structural issues contributing to engineering graduate student attrition. Coaching faculty in team management and encouraging close revision of departmental policies could help mitigate students' negative graduate experiences and decrease unnecessary attrition.
-
Objective: Identify the role(s) and support(s), if any, that family members provide to first-generation and historically marginalized doctoral students, including strengths and challenges of this support. Background: Nonfinancial family support is important for the success and retention of first-generation and historically marginalized graduate students. More empirical studies of the role(s) and support of family members of these doctoral students are needed. Method: During an intervention designed for firstgeneration and historically marginalized doctoral students and their families, we conducted four focus groups with doctoral students (n = 22) and three focus groups with the family members they chose to accompany them (n = 15). Transcripts were analyzed using thematic analysis. Results: Two themes emerged from the data: support and connection. In addition to providing distinct types of support, families play both supportive and connective roles. There are challenges to family roles and support in areas such as communication, doctoral student stress, and different ways that family members and doctoral students think about and approach life. Conclusion: The study provided key insights to understanding the roles and support of family of doctoral students; more efforts are needed across graduate schools in the United States. Implications: Family science faculty and graduate schools may collaborate to provide meaningful interventions for graduate students and their families for the goal of promoting graduate student retention and success.more » « less
-
Purpose This study aims to focus on the experiences of biomedical science students nearing the end of their doctoral programs and the factors that influence their well-being. In addition to identifying general challenges, the study aims to expand understanding of how interactions with principal investigators (PIs) can influence students’ well-being and engagement in wellness practices. Design/methodology/approach This qualitative study presents an analysis of interview data collected from 90 trainees five years after beginning their graduate programs. All were participants in a larger mixed-methods, longitudinal study. Emergent themes and a codebook were established after reviewing interview transcripts and completing memos. Codes were applied to data, and reports were generated to confirm and challenge early interpretations. Findings Participants described four key factors that influenced their well-being: perceived work/life balance; managing progress on research; program completion and job search; and overall faculty relationships. While relationships with PIs could be a source of stress, participants more often described how both interactions with, and observations of their PIs could amplify or mitigate their ability to manage other stressors and overall sense of well-being. Originality/value While researchers in the USA have increasingly considered the factors impacting graduate student mental health, there has been less of an emphasis on wellness and well-being. Furthermore, there has been less attention to how PIs contribute, in positive and negative ways, to these outcomes. This study offers insight into well-being at a specific timepoint, considering dynamics unique to wellness and well-being in the later stages of doctoral training.more » « less
