skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Evaluating STEM Broadening Participation Initiatives
A highly structured evaluation plan is essential to implementing a successful science, technology, engineering, and math (STEM) intervention program designed to improve diversity in STEM education and careers. Effective program evaluations facilitate program improvement and sustainability goals. This article reviews paramount program components and discusses the importance of integrating program components in designing a dynamic evaluation plan. Evaluation plan development must involve dialogue with program team members and participants before, during, and after the STEM intervention. Translation of raw data to actionable intelligence using appropriate analytical techniques is indispensable to maximize the insights derived from an effective evaluation plan.  more » « less
Award ID(s):
2205612
PAR ID:
10528300
Author(s) / Creator(s):
Date Published:
Journal Name:
International Education & Research Journal
Volume:
9
Issue:
7
ISSN:
2454-9916
Page Range / eLocation ID:
82-85
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Effective mentorship has been linked to improving research skills and productivity, reducing the risk of anxiety and depression, and maintaining students' academic persistence in STEM fields. Recognizing the enormous effect of mentoring on students’ academic careers and confidence, many universities and research institutions provide training programs to faculty to enhance their mentoring skills and knowledge and strengthen mentoring relationships. Many existing mentor programs for faculty adopt the evidence-based Entering Mentoring training curriculum, originally developed by a team of researchers and practitioners at the University of Wisconsin Madison for research faculty in STEM disciplines [1]. From the program evaluation perspective, measuring a training program's results and determining whether intended outcomes are achieved (i.e., the outcome evaluation) can tell whether a program achieves its goals. The results of the evaluation further inform program organizers of needed improvements. Therefore, having good knowledge of these assessment tools and their appropriate usage is essential to planning and evaluating mentoring programs. After the Entering Mentoring training curriculum was launched, various assessment tools and approaches have been utilized by different training program evaluations. This study systematically and thoroughly reviews the extant empirical studies in STEM fields (including intervention and program evaluation studies) that used the Entering Mentoring training curriculum since it was first introduced (2005 to 2023). The goal is to examine: (1) what outcome variables have been assessed to indicate the effectiveness of the Entering Mentoring training curriculum, and (2) what measurement instruments have been used to quantify the outcome variables? Finally, a further discussion is provided, particularly on selecting the appropriate tool based on research goals and resources. The findings of this study not only offer timely insight into research trends on the evaluation of the Entering Mentoring training curriculum in STEM fields but also guide professionals of mentoring training evaluation in identifying important outcome variables and selecting appropriate instrument tools. 
    more » « less
  2. This work in progress paper presents an overview of the Hispanic Alliance for the Graduate Education and the Professoriate (H-AGEP) program. H-AGEP is working on developing and implementing a new model to improve the preparation and transition of Hispanic STEM doctoral students into community college faculty positions. The partnership is a collaborative effort between the City College of New York (CCNY) (lead institution) and The University of Texas at El Paso (UTEP) along with a group of partner community colleges: LaGuardia Community College, Queensborough Community College, and El Paso Community College. The H-AGEP model consists of three main elements: (1) a training and mentoring program for effective STEM teaching at community colleges; (2) a training program for effective mentoring of community college students in STEM research; and (3) a professional development program to address career preparation, transitioning, and advancement at academic careers in community colleges. H-AGEP research goals are: (1) to consider the collected evaluation and research data to determine what intervention activities are most impactful, and (2) to better understand the career-decision making process of Hispanic STEM doctoral students regarding whether they will seek employment at community colleges and other two-year institutions. An interesting aspect of the partnership is that the institutions in El Paso, Texas, serve primarily a Mexican-American student population while the New York institutions serve primarily a Hispanic population of Caribbean origin. This provides the unique opportunity to compare Hispanic students from both groups. The program evaluation: (1) documents and provides feedback on H-AGEP activities and model implementation; and (2) assesses the extent to which H-AGEP is achieving its intended outcomes. Assessment results on the first cohort of students in the program show the value of including community college faculty as career and teaching mentors in the program. Furthermore, the effect of model interventions in students from the first cohort show positive advances in improving teaching skills, increasing student professional networks, and increasing interest and awareness in careers at community college. 
    more » « less
  3. The dramatic lack of diversity within science, technology, engineering, and mathematics (STEM) higher-education graduates is a serious issue facing the future of the STEM workforce. Colleges and universities are attempting to fix this disparity through targeted intervention programs aimed at increasing the persistence and retention of low-income, first-generation, and underrepresented (UR) students in STEM. A comprehensive synthesis of adopted strategies and analysis of their effectiveness is lacking. We conducted a systematic multiple-studies review, considering 31 empirical articles published between 2005 and 2020 that shared the outcomes of intentional intervention programs in place across the United States. We uncovered essential characteristics of successful programs and highlighted the top 10 critical program components to consider. These can be used to guide the development and execution of future programs’ support systems and interventions. This is a relatively new area of research, with most programs just in the earliest stages of implementation. It is essential to continue to follow the data on the outcomes of these programs, particularly their longitudinal impacts on the diversification of the STEM workforce, to be able to implement effective evidence-based practices. 
    more » « less
  4. Two cohorts of ten (10) first generation students from the local public school district have been recruited to an NSF S STEM scholarship program that provides navigational support in attending and graduating from a STEM-focused private university. The S-STEM funding complements a university scholarship to meet the full demonstrated need of each student for four years, including on-campus housing to ensure that our scholars can fully participate in the college experience. Intentional programming and a mentor network were implemented with an assets-based framework. Student surveys and program evaluation reveal that the scholarship program components that are the most effective and appreciated by the scholars are free summer courses, paid summer research experiences, and a “support team” that connects them to resources and assists them in navigating the university system. Also important to their sense of belonging at the university was the pre-orientation program (similar to a short bridge program) and their cohort of peers in the S-STEM program. Interviews with the S-STEM scholars were conducted alongside interviews with other first generation students not in the S-STEM program. The research study on student experiences revealed that a distributed and unconnected model of student support can be frustrating and ineffective for students. Instead, a core team of people that guides students to navigate the university system and to provide intentional programming at the appropriate times has emerged to be more effective. Thus, we have adapted our project to meet the needs of the students as we hear their stories and learn from them. To capture our students’ experiences and to engage them in co-designing inclusive college experiences within a supportive university system, we plan to have a design charrette with a graphic illustrator where the scholars will collectively share their stories and brainstorm ideas upon deliberate prompts. The facilitation will elevate multiple voices and reinforce learning and highlight interconnections. The graphic recording will translate the real-time conversations and the key ideas into a shared visual. The graphical artwork will serve as a visual representation of the voices of our scholars and serve as a tool to present what is possible for the university to redefine student experiences and set up systems for all students to succeed. Through this project, we aim to demonstrate and document the sufficient resources needed (e.g., human capital) to support the whole student, and in particular students in which the university system was not initially designed for. The findings provide a great opportunity for the university to strengthen student supports with the proper resources and systems for students, especially from historically underrepresented and marginalized groups. 
    more » « less
  5. Holton, Valerie (Ed.)
    This study focuses on the increasing disparities in STEM education achievement and long-term wage earnings of under-represented minority groups. As part of national efforts to improve the diversity of the STEM workforce, this study uses longitudinal data from the University of Houston’s Education Research Center (UH-ERC) to examine the effect of participation in a STEM focused intervention program (Houston-Louis Stokes Alliance for Minority Participation) on wage earnings across students from traditionally under-represented groups. Data analysis consisted of propensity score matching analysis, followed by an ordinal logistic regression model to measure program participation effects on wage earnings. Findings indicate a significant negative association between participation in STEM intervention program and long-term wage earnings. Results highlight the role of structural racism and human capital on perpetuating achievement and wage gaps across race and socio-economic status. Recommendations focus on career-preparedness as a tool to increase the human capital of under-represented groups, and institutional shifts in policy and program components that strive to reduce the impact of structural racism on this subpopulation of students. 
    more » « less