skip to main content


Title: Examining STEM Diagnostic Exam Scores and Self-efficacy as Predictors of Three-year STEM Psychological and Career Outcomes
In this research-based paper, we explore the relationships among Rice University STEM students’ high school preparation, psychological characteristics, and career aspirations. Although greater high school preparation in STEM coursework predicts higher STEM retention and performance in college [1], objective academic preparation and college performance do not fully explain STEM retention decisions, and the students who leave STEM are often not the lowest performing students [2]. Certain psychosocial experiences may also influence students’ STEM decisions. We explored the predictive validity of 1) a STEM diagnostic exam as an objective measure of high school science and math preparation and 2) self-efficacy as a psychological measure on long-term (three years later) STEM career aspirations and STEM identity of underprepared Rice STEM students. University administrators use diagnostic exam scores (along with other evidence of high school underpreparation) to identify students who might benefit from additional support. Using linear regression to explore the link between diagnostic exam scores and self-efficacy, exam scores predicted self-efficacy a semester after students’ first semester in college; exam scores were also marginally correlated with self-efficacy three years later. Early STEM career aspirations predicted later career aspirations, accounting for 21.3% of the variance of career outcome expectations three years later (β=.462, p=.006). Scores on the math diagnostic exam accounted for an additional 10.1% of the variance in students’ three-year STEM career aspirations (p=.041). Self-efficacy after students’ first semester did not predict future STEM aspirations. Early STEM identity explained 28.8% of the variance in three-year STEM identity (p=.001). Math diagnostic exam scores accounted for only marginal incremental variance after STEM identity, and self-efficacy after students’ first semester did not predict three-year STEM aspirations. Overall, we found that the diagnostic exam provided incremental predictive validity in STEM career aspirations after students’ sixth semester of college, indicating that early STEM preparation has long-lasting ramifications for students’ STEM career intentions. Our next steps include examining whether students’ diagnostic exam scores predict STEM graduation rates and final GPAs for science and math versus engineering majors.  more » « less
Award ID(s):
1565032
NSF-PAR ID:
10348878
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
American Society for Engineering Education
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. In this research-based paper, we discuss the development of a measure of Rice University students’ STEM study strategies and then explore the measure’s correlation with several important psychological outcomes in a sample of underprepared first-year STEM students (n=94). STEM attrition remains a pressing concern nationally, particularly for students who took less rigorous STEM courses in high school, a population that disproportionally comprises underrepresented minorities. The authors developed an 11-item measure of STEM-specific study strategies, termed the STEM Study Strategies Questionnaire. We explored STEM-specific identity, self-efficacy, and career aspirations, as well as perceived utility of attaining a STEM degree, using a model based on Eccles and Wigfield’s (2002) expectancy-value framework of achievement. An exploratory factor analysis found a four-factor solution to the newly developed scale: Group Work in STEM, Active STEM Learning, Interactions with STEM Professors, and STEM Exam Familiarity. The authors found significant moderate to strong correlations among all psychological variables, as well as with the Group Work and STEM Exam Familiarity factors. Next steps for this research are to develop further measure items to capture each of the four factors and to conduct confirmatory analyses on different samples of STEM students, both those who are relatively underprepared and appropriately prepared for college STEM coursework. 
    more » « less
  2. Measures of subject-related role identities in physics and math have been developed from research on the underlying constructs of identity in science education. The items for these measures capture three constructs of identity: students’ interest in the subject, students’ feeling of recognition by others, and students’ beliefs about their performance/competence in the subject area. In prior studies with late secondary and early post-secondary students, participants did not distinguish between performance beliefs (e.g., believing that they can do well in a particular subject) and competence beliefs (e.g., believing that they can understand a particular subject); therefore, performance/competence beliefs are measured as a single construct. These validated measures have been successful in predicting STEM career choices including physics, math, and engineering. Based on these measures of identity, literature on engineering identity, and my prior work on understanding engineering choice and belongingness through students’ science and math identities at the transition from high school to college, I developed a set of new engineering identity measures that capture and overall identification as an engineer, future engineering career identification, and students’ engineering-related interest, recognition, and performance/competence beliefs. I conducted a pilot survey of 371 first-year engineering students at three institutions within the U.S. during the spring semester of 2015. An exploratory factor analysis (EFA) was performed to examine the underlying structure of the piloted questions about students’ engineering identity. The measures loaded on three separate constructs that were consistent with the hypothesized constructs of interest, performance/competence and recognition. The developed items were used in a subsequent study deployed in the fall semester of 2015 that measured more than 2500 first-year engineering students’ attitudes and beliefs at four institutions within the U.S. The data on engineering identity measures from this second survey were analyzed using confirmatory factor analysis (CFA). The results indicated that the developed measures do extract a significant portion of the average variance in the latent constructs and the internal consistency of the measures (Cronbach’s α) falls within the acceptable and better range. The development of these items provides ways for engineering education researchers to more deeply explore the underlying self-beliefs in students’ engineering identity formation through quantitative measures with strong evidence for validity. 
    more » « less
  3. Background:

    The United States continues to invest considerable resources into developing the next generation of science, technology, engineering, and mathematics (STEM) talent. Efforts to shore up interest in pursuing STEM careers span decades and have increasingly focused on boosting interest among diverse student populations. Policymakers have called for engaging students in a greater STEM ecology of support that extends beyond the traditional classroom environment to increase student STEM career interest. Yet, few robust studies exist exploring the efficacy of many programmatic efforts and initiatives outside the regular curriculum intended to foster STEM interest. To maximize STEM education investments, promote wise policies, and help achieve the aim of creating STEM learning ecosystems that benefit diverse student populations and meet the nation’s STEM goals, it is crucial to examine the effectiveness of these kinds of STEM education initiatives in promoting STEM career aspirations.

    Purpose:

    The purpose of this quasi-experimental study was to examine the impact of one popular, yet understudied, STEM education initiative on students’ STEM career aspirations: participation in a university- or college-run STEM club or program activity (CPA) during high school. Specifically, we studied whether participation in a college-run STEM CPA at a postsecondary institution during high school was related to college-going students’ STEM career aspirations, and we examined whether that relationship differed depending on student characteristics and prior STEM interests.

    Research Design:

    We conducted a quasi-experimental investigation to explore the impact of participation in university- or college-run STEM CPAs on college-going students’ STEM career aspirations. We administered a retrospective cohort survey to students at 27 colleges and universities nationwide resulting in a sample of 15,847 respondents. An inverse probability of treatment weighted logistic regression model with a robust set of controls was computed to estimate the odds of expressing STEM career aspirations among those who participated in college-run STEM CPAs compared with the odds expressed among students who did not participate. Our weighting accounted for self-selection effects.

    Results:

    Quasi-experimental modeling results indicated that participation in university- or college-run STEM CPAs had a significant impact on the odds that college-going students would express STEM career aspirations relative to students who did not participate. The odds of expressing interest in a STEM career among participants in STEM CPAs were 1.49 times those of the control group. Robustness checks confirmed our results. The result held true for students whether or not they expressed interest in STEM careers prior to participation in STEM CPAs, and it held true across a diverse range of student characteristics (e.g., race, parental education, gender, standardized test scores, and family/school encouragement).

    Conclusions:

    Results suggest that university- and college-run STEM CPAs play an important role in the STEM education ecology, serving the national goal of expanding the pool of college-going students who aspire to STEM careers. Moreover, results showed that participation in university- and college-run STEM CPAs during high school is equally effective across diverse student characteristics. Policymakers, educators, and those charged with making investment decisions in STEM education should seriously consider university- and college-run STEM CPAs as a promising vehicle to promote diverse students’ STEM career aspirations in the broader STEM learning ecosystem and as an important complement to other STEM learning environments.

     
    more » « less
  4. The current study examines the validity of the RESP diagnostic exam and its predictive validity relative to standardized tests with a sample of students (N = 976) who matriculated into Rice University from 2012 to 2014. The RESP diagnostic exam was related to grades, and we found that the correlation between the RESP diagnostic exam and grades was greater for STEM grades than non-STEM grades. We found that the diagnostic exam accounted for an incremental 9% of variance in STEM grades above SAT performance, but only 1% of incremental variance above SAT in non-STEM grades. Moreover, we found evidence of range restriction for both SAT and RESP diagnostic exam performance for Rice University matriculants, further suggesting the utility of the diagnostic exam is at the lower end of the distribution. In summary, our results suggest that an additional diagnostic exam written by schools to specifically measure STEM preparation for their program can be a useful addition to procedures for selecting students for special experiences such as summer bridge programs. 
    more » « less
  5. Abstract

    Many social psychological variables, in addition to knowledge‐based factors such as academic preparedness, have been investigated individually as sources of the persistent gender gap in pSTEM (physical science, technology, engineering, and mathematics) fields. The present work tested all of these factors simultaneously and longitudinally, in a sample drawn from an incoming freshman class measured at three timepoints over the course of their first year. One thousand nine hundred and twenty‐nine students completed a survey with items assessing eleven social psychological constructs and were asked for permission to obtain institutional data regarding their academic preparedness and choice of academic major before matriculation, after the first semester, and at the end of freshman year. These social psychological and academic variables were used to predict pSTEM major status. Across multiple timepoints, and over and above academic preparedness, greater Math and Science Self‐Efficacy, rejection of the stereotype that scientist careers are unsociable in nature, having a pSTEM role model in high school, and lower endorsement of communal goals were consistently related to the selection of a pSTEM major. Students who endorsed entity theories of math and science abilities were also less likely to select a pSTEM over life and social sciences major. As a set, the factors accounted for roughly half of the gender difference in pSTEM major selection. Interventions aimed at reducing the gender gap in pSTEM major selection might do well to focus on these more psychological factors as well as academic ones.

     
    more » « less