Measuring Predictors of Persistence: An Instrument for First-Year Engineering Students

Currey, Matthew; Cruz, Juan  (ORCID:000000017426682X); Mallouk, Kaitlin  (ORCID:0000000343671165); Farrell, Stephanie; Major, Justin  (ORCID:0000000231118509)

doi:10.18260/1-2--60111

This project is part of a multi-year NSF-SSTEM intervention that seeks to increase graduation rates among engineering students, with a focus on low-income, academically talented engineering students. We created a first-year early intervention grounded in Tinto’s conceptual model of student motivation and persistence. This model defines the four contributors to students' motivation to persist in higher education as clarity of goals, self-efficacy, sense of belonging, and perceived curriculum relevance. In alignment with this framework, our interventions include scholarships, an Engineering Learning Community (ELC) to foster peer connections, mentorship programs, personal and professional development seminars, and inclusive practices integrated into first-year engineering lab courses. Together, these efforts aim to promote academic and social integration for first-year engineering students, thereby increasing their likelihood of persistence and graduation. In the first phase of the project, we developed and validated a survey instrument to measure whether these interventions influenced students’ experiences in ways consistent with Tinto’s model. Data collected in 2023 and 2024 from over 400 first-year engineering students at a four-year institution in the Northeastern U.S. underwent exploratory and confirmatory factor analyses. We found that the four predictors of persistence can be measured using six interrelated factors: (1) comfort with the professor, (2) sense of belonging, (3) engineering identity, (4) goals and intent to pursue, (5) test/assignment confidence, and (6) diversity, equity, and inclusion (DEI) components. Preliminary results provided promising validity evidence, though not all factors performed equally well. Comfort with professor emerged as a strong and stable construct, demonstrating high reliability across Cronbach’s alpha, McDonald’s omega, and Average Variance Extracted (AVE). Engineering identity also showed robust performance on these metrics. By contrast, the remaining four factors did not meet the desired benchmarks for internal consistency reliability and convergent validity. These findings highlighted the need to strengthen the instrument prior to broader implementation and generalization. The present study builds on this foundation with the explicit goal of strengthening validity evidence. To do so, we added targeted survey items to reinforce weaker constructs, thereby improving reliability metrics. We collected 363 responses from the current (2025-2026) cohort of first-year engineering students to increase statistical power and refine factor estimates. Finally, guided by Tinto’s framework and refined through input from students and faculty in our program, we introduced additional items that may establish a seventh factor: curriculum relevance. These items are likely to measure how students perceive the connection between coursework and their academic or career goals, a critical dimension of persistence not fully captured in the original instrument. The central research question guiding this study is: What is the internal reliability and convergent validity of our new enhanced instrument and how does it compare to the previous instrument? By addressing this question, we expect to enhance our instrument, enabling a more rigorous evaluation of first-year support interventions. In doing so, the project not only evaluates the effectiveness of our NSF-SSTEM program but also provides a model for institutions seeking evidence-based strategies to improve retention and graduation among first year engineering students.

More Like this