skip to main content


Title: Impact of Innovative Technology-Related Interventions on K–12 Students' STEM Career-Related Outcomes: A Meta-Analysis
This meta-analysis study reviews and synthesizes research and evaluation findings demonstrating the effects of integrating innovative technologies and technology-based learning experiences in STEM education on K-12 students’ STEM career-related outcomes. This study synthesizes a body of research from 1995 to the present, across characteristics of technology-based STEM education interventions, learning contexts, student demographics, and study designs. This study develops an understanding of the extent to which the effects of technology-based STEM education interventions are different for students who are traditionally underserved and underrepresented in STEM education. Findings suggest such interventions have small, positive effects on students’ STEM career outcomes. Interventions serving underrepresented students had larger influences on students’ career outcomes than those serving general student populations.  more » « less
Award ID(s):
1949437
NSF-PAR ID:
10382969
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
Impact of Innovative Technology-Related Interventions on K–12 Students' STEM Career-Related Outcomes: A Meta-Analysis
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Need/Motivation (e.g., goals, gaps in knowledge) The ESTEEM implemented a STEM building capacity project through students’ early access to a sustainable and innovative STEM Stepping Stones, called Micro-Internships (MI). The goal is to reap key benefits of a full-length internship and undergraduate research experiences in an abbreviated format, including access, success, degree completion, transfer, and recruiting and retaining more Latinx and underrepresented students into the STEM workforce. The MIs are designed with the goals to provide opportunities for students at a community college and HSI, with authentic STEM research and applied learning experiences (ALE), support for appropriate STEM pathway/career, preparation and confidence to succeed in STEM and engage in summer long REUs, and with improved outcomes. The MI projects are accessible early to more students and build momentum to better overcome critical obstacles to success. The MIs are shorter, flexibly scheduled throughout the year, easily accessible, and participation in multiple MI is encouraged. ESTEEM also establishes a sustainable and collaborative model, working with partners from BSCS Science Education, for MI’s mentor, training, compliance, and building capacity, with shared values and practices to maximize the improvement of student outcomes. New Knowledge (e.g., hypothesis, research questions) Research indicates that REU/internship experiences can be particularly powerful for students from Latinx and underrepresented groups in STEM. However, those experiences are difficult to access for many HSI-community college students (85% of our students hold off-campus jobs), and lack of confidence is a barrier for a majority of our students. The gap between those who can and those who cannot is the “internship access gap.” This project is at a central California Community College (CCC) and HSI, the only affordable post-secondary option in a region serving a historically underrepresented population in STEM, including 75% Hispanic, and 87% have not completed college. MI is designed to reduce inequalities inherent in the internship paradigm by providing access to professional and research skills for those underserved students. The MI has been designed to reduce barriers by offering: shorter duration (25 contact hours); flexible timing (one week to once a week over many weeks); open access/large group; and proximal location (on-campus). MI mentors participate in week-long summer workshops and ongoing monthly community of practice with the goal of co-constructing a shared vision, engaging in conversations about pedagogy and learning, and sustaining the MI program going forward. Approach (e.g., objectives/specific aims, research methodologies, and analysis) Research Question and Methodology: We want to know: How does participation in a micro-internship affect students’ interest and confidence to pursue STEM? We used a mixed-methods design triangulating quantitative Likert-style survey data with interpretive coding of open-responses to reveal themes in students’ motivations, attitudes toward STEM, and confidence. Participants: The study sampled students enrolled either part-time or full-time at the community college. Although each MI was classified within STEM, they were open to any interested student in any major. Demographically, participants self-identified as 70% Hispanic/Latinx, 13% Mixed-Race, and 42 female. Instrument: Student surveys were developed from two previously validated instruments that examine the impact of the MI intervention on student interest in STEM careers and pursuing internships/REUs. Also, the pre- and post (every e months to assess longitudinal outcomes) -surveys included relevant open response prompts. The surveys collected students’ demographics; interest, confidence, and motivation in pursuing a career in STEM; perceived obstacles; and past experiences with internships and MIs. 171 students responded to the pre-survey at the time of submission. Outcomes (e.g., preliminary findings, accomplishments to date) Because we just finished year 1, we lack at this time longitudinal data to reveal if student confidence is maintained over time and whether or not students are more likely to (i) enroll in more internships, (ii) transfer to a four-year university, or (iii) shorten the time it takes for degree attainment. For short term outcomes, students significantly Increased their confidence to continue pursuing opportunities to develop within the STEM pipeline, including full-length internships, completing STEM degrees, and applying for jobs in STEM. For example, using a 2-tailed t-test we compared means before and after the MI experience. 15 out of 16 questions that showed improvement in scores were related to student confidence to pursue STEM or perceived enjoyment of a STEM career. Finding from the free-response questions, showed that the majority of students reported enrolling in the MI to gain knowledge and experience. After the MI, 66% of students reported having gained valuable knowledge and experience, and 35% of students spoke about gaining confidence and/or momentum to pursue STEM as a career. Broader Impacts (e.g., the participation of underrepresented minorities in STEM; development of a diverse STEM workforce, enhanced infrastructure for research and education) The ESTEEM project has the potential for a transformational impact on STEM undergraduate education’s access and success for underrepresented and Latinx community college students, as well as for STEM capacity building at Hartnell College, a CCC and HSI, for students, faculty, professionals, and processes that foster research in STEM and education. Through sharing and transfer abilities of the ESTEEM model to similar institutions, the project has the potential to change the way students are served at an early and critical stage of their higher education experience at CCC, where one in every five community college student in the nation attends a CCC, over 67% of CCC students identify themselves with ethnic backgrounds that are not White, and 40 to 50% of University of California and California State University graduates in STEM started at a CCC, thus making it a key leverage point for recruiting and retaining a more diverse STEM workforce. 
    more » « less
  2. In 1991, the Texas A&M University System was one of the first six Louis Stokes Alliance for Minority Participation (LSAMP) awardees. In the three decades of programming, several high impact practices (HIP) have been emphasized. One of them, undergraduate research (UR), is discussed. All members of the Alliance are part of the Texas A&M University System and undergraduate research was supported through a variety of initiatives on the Alliance campuses. Data presented chronicle student perspectives. Topics addressed are the impact of involvement in undergraduate research on academic outcomes, interest in further engagement with research, interest in graduate school, and career goals as well as the patterns of research engagement participants experienced and the forms of learning that resulted. These materials are presented regarding an audience that was overwhelmingly underrepresented minority students all of whom were pursuing science, technology, engineering, or mathematics (STEM) degrees. Students reported UR influenced their academic outcomes, further engagement with research, interest in graduate school, and career goals while facilitating learning and skill development. These findings, for URM students from institutions with three different Carnegie classifications that are a predominantly white institution, two Hispanic-serving institutions (HSIs), and a historically Black college or university (HBCU), parallel outcomes reported in the literature for investigations focused on general student populations suggesting that UR benefits are generalizable regardless of institution type and ethnicity/race of the participant. Findings also suggest that these patterns apply regardless of the student’s year in school. Material presented details the research elements commonly included in TAMUS LSAMP UR experiences and in which areas students reported the most learning. Thus, this document touches on topics important in addressing development of an adequate, well-trained, and diverse STEM workforce. It also confirms the efficacy of a highly replicable approach to facilitating a HIP, undergraduate research, with students from underrepresented groups. 
    more » « less
  3. There is little research or understanding of curricular differences between two- and four-year programs, career development of engineering technology (ET) students, and professional preparation for ET early career professionals [1]. Yet, ET credentials (including certificates, two-, and four-year degrees) represent over half of all engineering credentials awarded in the U.S [2]. ET professionals are important hands-on members of engineering teams who have specialized knowledge of components and engineering systems. This research study focuses on how career orientations affect engineering formation of ET students educated at two-year colleges. The theoretical framework guiding this study is Social Cognitive Career Theory (SCCT). SCCT is a theory which situates attitudes, interests, and experiences and links self-efficacy beliefs, outcome expectations, and personal goals to educational and career decisions and outcomes [3]. Student knowledge of attitudes toward and motivation to pursue STEM and engineering education can impact academic performance and indicate future career interest and participation in the STEM workforce [4]. This knowledge may be measured through career orientations or career anchors. A career anchor is a combination of self-concept characteristics which includes talents, skills, abilities, motives, needs, attitudes, and values. Career anchors can develop over time and aid in shaping personal and career identity [6]. The purpose of this quantitative research study is to identify dimensions of career orientations and anchors at various educational stages to map to ET career pathways. The research question this study aims to answer is: For students educated in two-year college ET programs, how do the different dimensions of career orientations, at various phases of professional preparation, impact experiences and development of professional profiles and pathways? The participants (n=308) in this study represent three different groups: (1) students in engineering technology related programs from a medium rural-serving technical college (n=136), (2) students in engineering technology related programs from a large urban-serving technical college (n=52), and (3) engineering students at a medium Research 1 university who have transferred from a two-year college (n=120). All participants completed Schein’s Career Anchor Inventory [5]. This instrument contains 40 six-point Likert-scale items with eight subscales which correlate to the eight different career anchors. Additional demographic questions were also included. The data analysis includes graphical displays for data visualization and exploration, descriptive statistics for summarizing trends in the sample data, and then inferential statistics for determining statistical significance. This analysis examines career anchor results across groups by institution, major, demographics, types of educational experiences, types of work experiences, and career influences. This cross-group analysis aids in the development of profiles of values, talents, abilities, and motives to support customized career development tailored specifically for ET students. These findings contribute research to a gap in ET and two-year college engineering education research. Practical implications include use of findings to create career pathways mapped to career anchors, integration of career development tools into two-year college curricula and programs, greater support for career counselors, and creation of alternate and more diverse pathways into engineering. Words: 489 References [1] National Academy of Engineering. (2016). Engineering technology education in the United States. Washington, DC: The National Academies Press. [2] The Integrated Postsecondary Education Data System, (IPEDS). (2014). Data on engineering technology degrees. [3] Lent, R.W., & Brown, S.B. (1996). Social cognitive approach to career development: An overivew. Career Development Quarterly, 44, 310-321. [4] Unfried, A., Faber, M., Stanhope, D.S., Wiebe, E. (2015). The development and validation of a measure of student attitudes toward science, technology, engineeirng, and math (S-STEM). Journal of Psychoeducational Assessment, 33(7), 622-639. [5] Schein, E. (1996). Career anchors revisited: Implications for career development in the 21st century. Academy of Management Executive, 10(4), 80-88. [6] Schein, E.H., & Van Maanen, J. (2013). Career Anchors, 4th ed. San Francisco: Wiley. 
    more » « less
  4. At San Francisco State University, a Hispanic Serving Institute and a Primarily Undergraduate Institution, 67% of engineering students are from ethnic minority groups, with only 27% of Hispanic students retained and graduated in their senior year. Additionally, only 14% of students reported full-time employment secured at the time of graduation. Of these secured jobs, only 54% were full-time positions (40+ hours a week). To improve the situation, San Francisco State University, in collaboration with two local community colleges, Skyline and Cañada Colleges, was recently funded by the National Science Foundation through a Hispanic Serving Institute Improving Undergraduate STEM Education Strengthening Student Motivation and Resilience through Research and Advising program to enhance undergraduate engineering education and build capacity for student success. This project will use a data-driven and evidence-based approach to identify the barriers to the success of underrepresented minority students and to generate new knowledge on the best practices for increasing students’ retention and graduation rates, self- efficacy, professional development, and workforce preparedness. Three objectives underpin this overall goal. The first is to develop and implement a Summer Research Internship Program together with community college partners. The second is to establish an HSI Engineering Success Center to provide students with academic resources, networking opportunities with industry, and career development tools. The third is to develop resources for the professional development of faculty members, including Summer Faculty Teaching Workshops, an Inclusive Teaching and Mentoring Seminar Series, and an Engineering Faculty Learning Community. Qualitative and quantitative approaches are used to assess the project outcomes using a survey instrument and interview protocols developed by an external evaluator. This paper discusses an overview of the project and its first-year implementation. The focus is placed on the introduction and implementation of the several main project components, namely the Engineering Success Center, Summer Research Internship Program, and Faculty Summer Teaching Workshop. The preliminary evaluation results, demonstrating the great success of these strategies, are also discussed. 
    more » « less
  5. Background:

    The United States continues to invest considerable resources into developing the next generation of science, technology, engineering, and mathematics (STEM) talent. Efforts to shore up interest in pursuing STEM careers span decades and have increasingly focused on boosting interest among diverse student populations. Policymakers have called for engaging students in a greater STEM ecology of support that extends beyond the traditional classroom environment to increase student STEM career interest. Yet, few robust studies exist exploring the efficacy of many programmatic efforts and initiatives outside the regular curriculum intended to foster STEM interest. To maximize STEM education investments, promote wise policies, and help achieve the aim of creating STEM learning ecosystems that benefit diverse student populations and meet the nation’s STEM goals, it is crucial to examine the effectiveness of these kinds of STEM education initiatives in promoting STEM career aspirations.

    Purpose:

    The purpose of this quasi-experimental study was to examine the impact of one popular, yet understudied, STEM education initiative on students’ STEM career aspirations: participation in a university- or college-run STEM club or program activity (CPA) during high school. Specifically, we studied whether participation in a college-run STEM CPA at a postsecondary institution during high school was related to college-going students’ STEM career aspirations, and we examined whether that relationship differed depending on student characteristics and prior STEM interests.

    Research Design:

    We conducted a quasi-experimental investigation to explore the impact of participation in university- or college-run STEM CPAs on college-going students’ STEM career aspirations. We administered a retrospective cohort survey to students at 27 colleges and universities nationwide resulting in a sample of 15,847 respondents. An inverse probability of treatment weighted logistic regression model with a robust set of controls was computed to estimate the odds of expressing STEM career aspirations among those who participated in college-run STEM CPAs compared with the odds expressed among students who did not participate. Our weighting accounted for self-selection effects.

    Results:

    Quasi-experimental modeling results indicated that participation in university- or college-run STEM CPAs had a significant impact on the odds that college-going students would express STEM career aspirations relative to students who did not participate. The odds of expressing interest in a STEM career among participants in STEM CPAs were 1.49 times those of the control group. Robustness checks confirmed our results. The result held true for students whether or not they expressed interest in STEM careers prior to participation in STEM CPAs, and it held true across a diverse range of student characteristics (e.g., race, parental education, gender, standardized test scores, and family/school encouragement).

    Conclusions:

    Results suggest that university- and college-run STEM CPAs play an important role in the STEM education ecology, serving the national goal of expanding the pool of college-going students who aspire to STEM careers. Moreover, results showed that participation in university- and college-run STEM CPAs during high school is equally effective across diverse student characteristics. Policymakers, educators, and those charged with making investment decisions in STEM education should seriously consider university- and college-run STEM CPAs as a promising vehicle to promote diverse students’ STEM career aspirations in the broader STEM learning ecosystem and as an important complement to other STEM learning environments.

     
    more » « less