skip to main content

Title: Examining STEM Diagnostic Exam Scores and Self-efficacy as Predictors of Three-year STEM Psychological and Career Outcomes
In this research-based paper, we explore the relationships among Rice University STEM students’ high school preparation, psychological characteristics, and career aspirations. Although greater high school preparation in STEM coursework predicts higher STEM retention and performance in college [1], objective academic preparation and college performance do not fully explain STEM retention decisions, and the students who leave STEM are often not the lowest performing students [2]. Certain psychosocial experiences may also influence students’ STEM decisions. We explored the predictive validity of 1) a STEM diagnostic exam as an objective measure of high school science and math preparation and 2) self-efficacy as a psychological measure on long-term (three years later) STEM career aspirations and STEM identity of underprepared Rice STEM students. University administrators use diagnostic exam scores (along with other evidence of high school underpreparation) to identify students who might benefit from additional support. Using linear regression to explore the link between diagnostic exam scores and self-efficacy, exam scores predicted self-efficacy a semester after students’ first semester in college; exam scores were also marginally correlated with self-efficacy three years later. Early STEM career aspirations predicted later career aspirations, accounting for 21.3% of the variance of career outcome expectations three years later (β=.462, p=.006). Scores more » on the math diagnostic exam accounted for an additional 10.1% of the variance in students’ three-year STEM career aspirations (p=.041). Self-efficacy after students’ first semester did not predict future STEM aspirations. Early STEM identity explained 28.8% of the variance in three-year STEM identity (p=.001). Math diagnostic exam scores accounted for only marginal incremental variance after STEM identity, and self-efficacy after students’ first semester did not predict three-year STEM aspirations. Overall, we found that the diagnostic exam provided incremental predictive validity in STEM career aspirations after students’ sixth semester of college, indicating that early STEM preparation has long-lasting ramifications for students’ STEM career intentions. Our next steps include examining whether students’ diagnostic exam scores predict STEM graduation rates and final GPAs for science and math versus engineering majors. « less
; ; ;
Award ID(s):
Publication Date:
Journal Name:
American Society for Engineering Education
Sponsoring Org:
National Science Foundation
More Like this
  1. In this research-based paper, we discuss the development of a measure of Rice University students’ STEM study strategies and then explore the measure’s correlation with several important psychological outcomes in a sample of underprepared first-year STEM students (n=94). STEM attrition remains a pressing concern nationally, particularly for students who took less rigorous STEM courses in high school, a population that disproportionally comprises underrepresented minorities. The authors developed an 11-item measure of STEM-specific study strategies, termed the STEM Study Strategies Questionnaire. We explored STEM-specific identity, self-efficacy, and career aspirations, as well as perceived utility of attaining a STEM degree, using a model based on Eccles and Wigfield’s (2002) expectancy-value framework of achievement. An exploratory factor analysis found a four-factor solution to the newly developed scale: Group Work in STEM, Active STEM Learning, Interactions with STEM Professors, and STEM Exam Familiarity. The authors found significant moderate to strong correlations among all psychological variables, as well as with the Group Work and STEM Exam Familiarity factors. Next steps for this research are to develop further measure items to capture each of the four factors and to conduct confirmatory analyses on different samples of STEM students, both those who are relatively underprepared and appropriatelymore »prepared for college STEM coursework.« less
  2. Measures of subject-related role identities in physics and math have been developed from research on the underlying constructs of identity in science education. The items for these measures capture three constructs of identity: students’ interest in the subject, students’ feeling of recognition by others, and students’ beliefs about their performance/competence in the subject area. In prior studies with late secondary and early post-secondary students, participants did not distinguish between performance beliefs (e.g., believing that they can do well in a particular subject) and competence beliefs (e.g., believing that they can understand a particular subject); therefore, performance/competence beliefs are measured as a single construct. These validated measures have been successful in predicting STEM career choices including physics, math, and engineering. Based on these measures of identity, literature on engineering identity, and my prior work on understanding engineering choice and belongingness through students’ science and math identities at the transition from high school to college, I developed a set of new engineering identity measures that capture and overall identification as an engineer, future engineering career identification, and students’ engineering-related interest, recognition, and performance/competence beliefs. I conducted a pilot survey of 371 first-year engineering students at three institutions within the U.S. during themore »spring semester of 2015. An exploratory factor analysis (EFA) was performed to examine the underlying structure of the piloted questions about students’ engineering identity. The measures loaded on three separate constructs that were consistent with the hypothesized constructs of interest, performance/competence and recognition. The developed items were used in a subsequent study deployed in the fall semester of 2015 that measured more than 2500 first-year engineering students’ attitudes and beliefs at four institutions within the U.S. The data on engineering identity measures from this second survey were analyzed using confirmatory factor analysis (CFA). The results indicated that the developed measures do extract a significant portion of the average variance in the latent constructs and the internal consistency of the measures (Cronbach’s α) falls within the acceptable and better range. The development of these items provides ways for engineering education researchers to more deeply explore the underlying self-beliefs in students’ engineering identity formation through quantitative measures with strong evidence for validity.« less
  3. Background:

    The United States continues to invest considerable resources into developing the next generation of science, technology, engineering, and mathematics (STEM) talent. Efforts to shore up interest in pursuing STEM careers span decades and have increasingly focused on boosting interest among diverse student populations. Policymakers have called for engaging students in a greater STEM ecology of support that extends beyond the traditional classroom environment to increase student STEM career interest. Yet, few robust studies exist exploring the efficacy of many programmatic efforts and initiatives outside the regular curriculum intended to foster STEM interest. To maximize STEM education investments, promote wise policies, and help achieve the aim of creating STEM learning ecosystems that benefit diverse student populations and meet the nation’s STEM goals, it is crucial to examine the effectiveness of these kinds of STEM education initiatives in promoting STEM career aspirations.


    The purpose of this quasi-experimental study was to examine the impact of one popular, yet understudied, STEM education initiative on students’ STEM career aspirations: participation in a university- or college-run STEM club or program activity (CPA) during high school. Specifically, we studied whether participation in a college-run STEM CPA at a postsecondary institution during high school was related to college-goingmore »students’ STEM career aspirations, and we examined whether that relationship differed depending on student characteristics and prior STEM interests.

    Research Design:

    We conducted a quasi-experimental investigation to explore the impact of participation in university- or college-run STEM CPAs on college-going students’ STEM career aspirations. We administered a retrospective cohort survey to students at 27 colleges and universities nationwide resulting in a sample of 15,847 respondents. An inverse probability of treatment weighted logistic regression model with a robust set of controls was computed to estimate the odds of expressing STEM career aspirations among those who participated in college-run STEM CPAs compared with the odds expressed among students who did not participate. Our weighting accounted for self-selection effects.


    Quasi-experimental modeling results indicated that participation in university- or college-run STEM CPAs had a significant impact on the odds that college-going students would express STEM career aspirations relative to students who did not participate. The odds of expressing interest in a STEM career among participants in STEM CPAs were 1.49 times those of the control group. Robustness checks confirmed our results. The result held true for students whether or not they expressed interest in STEM careers prior to participation in STEM CPAs, and it held true across a diverse range of student characteristics (e.g., race, parental education, gender, standardized test scores, and family/school encouragement).


    Results suggest that university- and college-run STEM CPAs play an important role in the STEM education ecology, serving the national goal of expanding the pool of college-going students who aspire to STEM careers. Moreover, results showed that participation in university- and college-run STEM CPAs during high school is equally effective across diverse student characteristics. Policymakers, educators, and those charged with making investment decisions in STEM education should seriously consider university- and college-run STEM CPAs as a promising vehicle to promote diverse students’ STEM career aspirations in the broader STEM learning ecosystem and as an important complement to other STEM learning environments.

    « less
  4. The current study examines the validity of the RESP diagnostic exam and its predictive validity relative to standardized tests with a sample of students (N = 976) who matriculated into Rice University from 2012 to 2014. The RESP diagnostic exam was related to grades, and we found that the correlation between the RESP diagnostic exam and grades was greater for STEM grades than non-STEM grades. We found that the diagnostic exam accounted for an incremental 9% of variance in STEM grades above SAT performance, but only 1% of incremental variance above SAT in non-STEM grades. Moreover, we found evidence of range restriction for both SAT and RESP diagnostic exam performance for Rice University matriculants, further suggesting the utility of the diagnostic exam is at the lower end of the distribution. In summary, our results suggest that an additional diagnostic exam written by schools to specifically measure STEM preparation for their program can be a useful addition to procedures for selecting students for special experiences such as summer bridge programs.
  5. There is little research or understanding of curricular differences between two- and four-year programs, career development of engineering technology (ET) students, and professional preparation for ET early career professionals [1]. Yet, ET credentials (including certificates, two-, and four-year degrees) represent over half of all engineering credentials awarded in the U.S [2]. ET professionals are important hands-on members of engineering teams who have specialized knowledge of components and engineering systems. This research study focuses on how career orientations affect engineering formation of ET students educated at two-year colleges. The theoretical framework guiding this study is Social Cognitive Career Theory (SCCT). SCCT is a theory which situates attitudes, interests, and experiences and links self-efficacy beliefs, outcome expectations, and personal goals to educational and career decisions and outcomes [3]. Student knowledge of attitudes toward and motivation to pursue STEM and engineering education can impact academic performance and indicate future career interest and participation in the STEM workforce [4]. This knowledge may be measured through career orientations or career anchors. A career anchor is a combination of self-concept characteristics which includes talents, skills, abilities, motives, needs, attitudes, and values. Career anchors can develop over time and aid in shaping personal and career identity [6].more »The purpose of this quantitative research study is to identify dimensions of career orientations and anchors at various educational stages to map to ET career pathways. The research question this study aims to answer is: For students educated in two-year college ET programs, how do the different dimensions of career orientations, at various phases of professional preparation, impact experiences and development of professional profiles and pathways? The participants (n=308) in this study represent three different groups: (1) students in engineering technology related programs from a medium rural-serving technical college (n=136), (2) students in engineering technology related programs from a large urban-serving technical college (n=52), and (3) engineering students at a medium Research 1 university who have transferred from a two-year college (n=120). All participants completed Schein’s Career Anchor Inventory [5]. This instrument contains 40 six-point Likert-scale items with eight subscales which correlate to the eight different career anchors. Additional demographic questions were also included. The data analysis includes graphical displays for data visualization and exploration, descriptive statistics for summarizing trends in the sample data, and then inferential statistics for determining statistical significance. This analysis examines career anchor results across groups by institution, major, demographics, types of educational experiences, types of work experiences, and career influences. This cross-group analysis aids in the development of profiles of values, talents, abilities, and motives to support customized career development tailored specifically for ET students. These findings contribute research to a gap in ET and two-year college engineering education research. Practical implications include use of findings to create career pathways mapped to career anchors, integration of career development tools into two-year college curricula and programs, greater support for career counselors, and creation of alternate and more diverse pathways into engineering. Words: 489 References [1] National Academy of Engineering. (2016). Engineering technology education in the United States. Washington, DC: The National Academies Press. [2] The Integrated Postsecondary Education Data System, (IPEDS). (2014). Data on engineering technology degrees. [3] Lent, R.W., & Brown, S.B. (1996). Social cognitive approach to career development: An overivew. Career Development Quarterly, 44, 310-321. [4] Unfried, A., Faber, M., Stanhope, D.S., Wiebe, E. (2015). The development and validation of a measure of student attitudes toward science, technology, engineeirng, and math (S-STEM). Journal of Psychoeducational Assessment, 33(7), 622-639. [5] Schein, E. (1996). Career anchors revisited: Implications for career development in the 21st century. Academy of Management Executive, 10(4), 80-88. [6] Schein, E.H., & Van Maanen, J. (2013). Career Anchors, 4th ed. San Francisco: Wiley.« less