An official website of the United States government
Aim/Purpose: The research reported here aims to demonstrate a method by which novel applications of qualitative data in quantitative research can resolve ceiling effect tensions for educational and psychological research.Background: Self-report surveys and scales are essential to graduate education and social science research. Ceiling effects reflect the clustering of responses at the highest response categories resulting in non-linearity, a lack of variability which inhibits and distorts statistical analyses. Ceiling effects in stress reported by students can negatively impact the accuracy and utility of the resulting data.Methodology: A longitudinal sample example from graduate engineering students’ stress, open-ended critical events, and their early departure from doctoral study considerations demonstrate the utility and improved accuracy of adjusted stress measures to include open-ended critical event responses. Descriptive statistics are used to describe the ceiling effects in stress data and adjusted stress data. The longitudinal stress ratings were used to predict departure considerations in multilevel modeling ANCOVA analyses and demonstrate improved model predictiveness.Contribution: Combining qualitative data from open-ended responses with quantitative survey responses provides an opportunity to reduce ceiling effects and improve model performance in predicting graduate student persistence. Here, we present a method for adjusting stress scale responses by incorporating coded critical events based on the Taxonomy of Life Events, the application of this method in the analysis of stress responses in a longitudinal data set, and potential applications.Findings: The resulting process more effectively represents the doctoral student experience within statistical analyses. Stress and major life events significantly impact engineering doctoral students’ departure considerations.Recommendations for Practitioners: Graduate educators should be aware of students’ life events and assist students in managing graduate school expectations while maintaining progress toward their degree. Recommendation for Researchers: Integrating coded open-ended qualitative data into statistical models can increase the accuracy and representation of the lived student experience. The new approach improves the accuracy and presentation of students’ lived experiences by incorporating qualitative data into longitudinal analyses. The improvement assists researchers in correcting data with ceiling effects for use in longitudinal analyses.Impact on Society: The method described here provides a framework to systematically include open-ended qualitative data in which ceiling effects are present.Future Research: Future research should validate the coding process in similar samples and in samples of doctoral students in different fields and master’s students.
more »
« less
A Novel Research Design: Using Multilevel Discrete-Time Survival Analysis to Investigate the Effect of Calculus I on Engineering Student Persistence
The persistence of engineering students through graduation continues to be a concern in higher education. Previous studies have highlighted a link between students' performance in introductory mathematics courses and graduation rates. Focusing on a crucial foundational course within the engineering curriculum, the purpose of this study is to investigate how students’ performance in Calculus I impact their persistence in the engineering program. Utilizing data from 22 diverse educational institutions using Multiple-Institution Database for Investigating Engineering Longitudinal Development (MIDFIELD), multilevel discrete-time survival analysis is employed to examine the longitudinal persistence patterns among the nested structure of students within institutions. Discrete-time analysis is an event-based approach that has the advantage of analyzing time in discrete chunks during which the event of interest could occur. The technique is a type of survival analysis, which has been used in other studies in engineering education and other educational studies. This approach addresses various challenges associated with analyzing student persistence data such as dealing with censored observations – observations for whom their entire educational pathway is not yet known because they are still enrolled. Using a multilevel form of this analysis approach also accounts for the hierarchical nature of the data involving students nested within institutions and incorporating variables that change over time. Thus, the study takes into account the variability and complexities inherent in the analysis of different institutions and examines persistence patterns more comprehensively than previous studies. By incorporating a diverse range of institutions, the study captures a broader spectrum of experiences and contexts, which enhances the generalizability of the results.
more »
« less
- Award ID(s):
- 2142087
- PAR ID:
- 10587597
- Publisher / Repository:
- ASEE Conferences
- Date Published:
- Format(s):
- Medium: X
- Location:
- Portland, Oregon
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This research to practice paper describes two NSF-funded projects: The “Program for Engineering Access, Retention, and LIATS Success” (PEARLS) and the “Academic and Socioemotional Support Ecosystem for Talented low-income STEM Students” (ASSETS). These initiatives aim to support low-income, academically talented students pursuing STEM degrees, offering academic, socio-emotional, and financial assistance. Both projects provide mentoring, workshops, mental health services, and scholarships. Outcomes have shown increased engagement, retention, and graduation rates among participants compared to non-participants. Retention and persistence rates among PEARLS participants reached 97% and 96.3% respectively, showcasing the model effectiveness in keeping students focused on their studies. Graduation rates saw remarkable improvements, with on-time graduation rates soaring to 4.17 times that of non-participating students. The paper presents a robust framework for an integrated support ecosystem, emphasizing scalability and replicability across diverse institutions. By addressing the challenges faced by underprivileged students during environmental crises, this framework aims to foster equity and inclusion in higher education, ensuring all students have the support they need to succeed.more » « less
-
Multiple stakeholders are interested in measuring undergraduate student success in college across academic fields. Different metrics might appeal to different stakeholders. Some metrics such as the fraction of first-time, full-time students who start in the fall who graduate within six years, the graduation rate, are federally mandated by the U.S. Department of Education, Integrated Postsecondary Education Data System (IPEDS). We argue that this calculation of graduation rate is inherently problematic because it excludes up to 60% of students who transfer into an institution, enroll part-time, or enroll in terms other than the fall. By expanding the starters definition, we propose a graduation rate definition that includes conventionally excluded students and provides information on progression in a specific program. Stickiness is an even more-inclusive alternative, measuring a program’s success in graduating all undergraduates ever enrolled in the program. In this work, programs are grouped into six academic fields: Arts and Humanities, Business, Engineering, Other, Social Sciences, and STM (Science, Technology, and Mathematics. Stickiness is the percentage of students who ever enroll in an academic field that graduate in the same field. We use the Multiple Institution Dataset for Investigating Engineering Longitudinal Development (MIDFIELD) 2023 which contains unit-record data for over 2 million individual students at 19 institutions. For the academic fields studied, Engineering has the highest graduation rate and third highest stickiness. Social Sciences and Business also have higher graduation rates and stickiness than the other fields. We also track the relative fraction of students migrating to and from each academic field. This paper continues our work to derive better metrics for understanding student success.more » « less
-
Longitudinal analysis of nationwide single and multi-institutional data shows the positive relationship between student educational outcomes and a diverse student population. Various position papers and empirical studies have raised awareness about the importance of diversity in higher education within the academic community and policy makers over the past half century. However, lack of participation by underrepresented students in higher education remains a chronic and multidimensional problem. Mitigating any particular factor and expecting broad based impact has not worked and will not work. The U.S. Department of education suggested some proven, over-arching principles for institutions of higher education to increase diversity, viz.: institutional commitment, diversity at all levels, outreach and recruitment, support services for students, and an inclusive campus environment. While some of these principles can only be addressed at the institutional level, a department or college can adopt scaled versions of these principles and influence the policies at the institutional level. This paper discusses the journey of a school of engineering towards developing strategies for improving equity, inclusion, and diversity in the graduate programs in engineering. In the process, this group of researchers articulated some critical issues that prevent diverse and economically disadvantaged undergraduate students from seeking a graduate degree in engineering. The authors have identified the following major reasons hindering students from pursuing a graduate degree: lack of financial support and resources, fear of the unknown, imposter syndrome, and family pressure to start earning as soon as possible. Each of these areas requires a targeted approach to help diversify the graduate engineering programs. A GVSU team comprised of administrators and faculty members sought to build a comprehensive program that incorporates all of the aforementioned structures and others. This paper describes the development strategy of such a program that culminated with an NSF (National Science Foundation) award.more » « less
-
Over the last decade, the emergence of technician education has taken center stage at secondary and post-secondary educational institutions through the proliferation of industry needs and employer-partner engagement. Through employer-partner engagement, Southern University at Shreveport, Louisiana (SUSLA), a community college unit within the Southern University and A&M College System, conducted a systematic review, assessment, and revision of its curricula that addressed the needs of industry. SUSLA’s comprehensive assessment, guided by employer-partner input, community college and 4-year university collaborations, and published workforce data, identified a pressing need to develop an Engineering Technology program. To that end, SUSLA developed a 2+2+2 Matriculation Model within the Engineering Technology associate of applied science degree program designed as a gateway to enable early education, persistence to post-secondary credentials of value, and high-quality career outcomes. Programs with similar demographics may be able to use this as a model which aims to do four things: 1) facilitate the early engagement of students decreasing the number of academically underprepared learners entering college, 2) expand postsecondary educational opportunities to improve outcomes fostering economic opportunity 3) increase the enrollment, persistence and graduation of early education and underserved populations in STEM and, 4) facilitate a reduction in time to degree. SUSLA’s 2+2+2 Matriculation Model provides high school students with the opportunity to earn dual-enrollment post-secondary credentials through its certificate of technical offerings in Engineering and Engineering Technology. Additionally, the model facilitates an increase in the participation rate of students in STEM and provides more academic opportunities and career exploration through collaborative industry-academic networks. Furthermore, the model produces workforce-ready technicians which accelerates academic and technical skill attainment and, thereby addresses a) the shortage in moderate to middle-skill jobs that require some college, but less than a bachelor’s degree; and b) the ongoing need for highly skilled STEM graduates to maintain the nation’s competitiveness in productivity and innovation.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.18260/1-2--46469
