An official website of the United States government
Informal science education researchers have become increasingly interested in how out-of-school spaces that offer STEM (science, technology, engineering, and math) programs inform learners’ STEM achievement, interests, and affective outcomes. Studies have found that these spaces can offer critical learning and developmental opportunities for underrepresented racially minoritized (URM) students (Black, Latinx, low socioeconomic status) in STEM subjects. Shifting away from the leaky STEM pipeline analogy, researchers have posited contemporary understandings to explain why the minoritization of URM girls persists. Informal learning environments such as STEM summer camps are being studied to assess how URM girls experience and interact with STEM in novel ways. These environments can inform the research field about how URM girls’ perceptions of their STEM identities, abilities, efficacy, and belonging in STEM develop as they engage in those spaces. This mixed-method study used a multiple-case-study approach to examine how aspects of URM middle school girls’ STEM identities positively changed after participating in a one-week, sleep-away, single-gender STEM summer camp held at a university in the Southwestern U.S. Drawing on intersectionality and STEM identity, we used ecological systems theory to design our research study, examining how URM middle school girls narrate their STEM identities in this informal learning environment. Using quantitative analyses and deductive coding methods, we explored how elements of girls’ STEM identities were shaped during and after their participation in the STEM summer camp. Findings from our study highlight (1) quantitative changes in girl participants’ STEM identities, sense of belonging in STEM, and perceived STEM ability belief, (2) qualitative results supporting our quantitative findings, and (3) how the intersectionality of participants’ race and gender played a role in their STEM identities. This study points to the potential of STEM informal learning camps as a way of developing and fostering URM girls’ STEM identities.
more »
« less
Practical Rubrics for Informal Science Education Studies: (1) a STEM Research Design Rubric for Assessing Study Design and a (2) STEM Impact Rubric for Measuring Evidence of Impact
Informal learning institutions, such as museums, science centers, and community-based organizations, play a critical role in providing opportunities for students to engage in science, technology, engineering, and mathematics (STEM) activities during out-of-school time hours. In recent years, thousands of studies, evaluations, and conference proceedings have been published measuring the impact that these programs have had on their participants. However, because studies of informal science education (ISE) programs vary considerably in how they are designed and in the quality of their designs, it is often quite difficult to assess their impact on participants. Knowing whether the outcomes reported by these studies are supported with sufficient evidence is important not only for maximizing participant impact, but also because there are considerable economic and human resources invested to support informal learning initiatives. To address this problem, I used the theories of impact analysis and triangulation as a framework for developing user-friendly rubrics for assessing quality of research designs and evidence of impact. I used two main sources, research-based recommendations from STEM governing bodies and feedback from a focus group, to identify criteria indicative of high-quality STEM research and study design. Accordingly, I developed three STEM Research Design Rubrics, one for quantitative studies, one for qualitative studies, and another for mixed methods studies, that can be used by ISE researchers, practitioners, and evaluators to assess research design quality. Likewise, I developed three STEM Impact Rubrics, one for quantitative studies, one for qualitative studies, and another for mixed methods studies, that can be used by ISE researchers, practitioners, and evaluators to assess evidence of outcomes. The rubrics developed in this study are practical tools that can be used by ISE researchers, practitioners, and evaluators to improve the field of informal science learning by increasing the quality of study design and for discerning whether studies or program evaluations are providing sufficient evidence of impact.
more »
« less
- Award ID(s):
- 1710792
- PAR ID:
- 10224889
- Date Published:
- Journal Name:
- Frontiers in Education
- Volume:
- 5
- ISSN:
- 2504-284X
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Research has documented the presence of bias against women in hiring, including in academic science, technology, engineering, and mathematics (STEM). Hiring rubrics (also called criterion checklists, decision support tools, and evaluation tools) are widely recommended as a precise, cost-effective remedy to counteract hiring bias, despite a paucity of evidence that they actually work (see table S8). Our in-depth case study of rubric usage in faculty hiring in an academic engineering department in a very research-active university found that the rate of hiring women increased after the department deployed rubrics and used them to guide holistic discussions. Yet we also found evidence of substantial gender bias persisting in some rubric scoring categories and evaluators’ written comments. We do not recommend abandoning rubrics. Instead, we recommend a strategic and sociologically astute use of rubrics as a department self-study tool within the context of a holistic evaluation of semifinalist candidates.more » « less
-
Abstract This meta-analysis explores the impact of informal science education experiences (such as after-school programs, enrichment activities, etc.) on students' attitudes towards, and interest in, STEM disciplines (Science, Technology, Engineering, and Mathematics). The research addresses two primary questions: (1) What is the overall effect size of informal science learning experiences on students' attitudes towards and interest in STEM? (2) How do various moderating factors (e.g., types of informal learning experience, student grade level, academic subjects, etc.) impact student attitudes and interests in STEM? The studies included in this analysis were conducted within the United States in K-12 educational settings, over a span of thirty years (1992–2022). The findings indicate a positive association between informal science education programs and student interest in STEM. Moreover, the variability in these effects is contingent upon several moderating factors, including the nature of the informal science program, student grade level, STEM subjects, publication type, and publication year. Summarized effects of informal science education on STEM interest are delineated, and the implications for research, pedagogy, and practice are discussed.more » « less
-
Usable STEM knowledge for tomorrow's STEM problems More universities and education programs need more STEM knowledge in formal and informal settings to guide learners in applying STEM learning to the creation of solutions. To address this challenge, Nancy Butler Songer, the dean of the College of Education at the University of Utah designed a learning approach, Solutioning, that guides youth to deepen science content through science and engineering practices. Creating a six-week curricular program, the learning approach provided opportunities for students to use engineering design to create and provide feedback on a trap design that would attract a local invasive insect that was harmful to their community. Research was conducted on studies to provide empirical evidence on student STEM knowledge and learning and their ability to define science and engineering. Research results indicate that even elementary-age students demonstrate significant improvement in their understanding of STEM arguments as evaluated with a pre-post assessment before and after implementing a six-week solutioning curricular program.more » « less
-
Industry leaders emphasize that engineering students' technical communication and writing skills must be improved. Despite various institutional efforts, which include technical communication courses or engineering design projects aimed at enhancing students’ communication abilities, many believe there has been only slow improvement in this area. There has also been a dearth of longitudinal studies that examine the development of engineering students’ technical communication competencies from undergraduate to industry. This paper aims to contribute to this area through the creation of a rubric that specifically examines the writing competencies and technical communication ability of engineering students. This paper is part of a larger, NSF-funded research study that examines the quality of students’ written and oral communication skills and seeks to understand their relationship to the students’ spatial abilities. First-year engineering students in their second semester at a large R1 Midwestern university were examined. Students were tasked with creating a written report responding to a set of questions that asked about their team-based engineering design project completed in their first semester. As this occurred months prior, this non-graded report became a reflection on their experience and innate abilities. While low stakes, it mimicked a more authentic writing experience students encounter in industry. Students' responses were examined collaboratively by an interdisciplinary team which created a rubric through an iterative process. This rubric was distributed to the interdisciplinary team and outside evaluators composed of individuals in industry and engineering faculty. An inter-rater reliability analysis was conducted to examine levels of agreement between the interdisciplinary team and outside evaluators, and implications of this inter-rater reliability score and the process of rubric application were documented. Results of this paper include details on the development of a rubric that examine students’ technical communication and writing skills. Traditional rubrics utilized by engineering faculty usually address an entire project for engineering students, which includes students' content knowledge, writing capabilities, and the requirements of the project. Such rubrics are often used to provide feedback to students and evaluation in the form of grades. The narrower focus of the rubric being developed here can provide insights into communication and writing competencies of engineering students. Scores secured through the use of this rubric will aid in the research study’s goal of finding correlations between engineering students’ communication skills and spatial abilities (assessed outside of this current effort). Spatial ability has been well-documented as an effective indicator of success in STEM, and interventions have been developed to support development in students with weaker spatial skills. 23, 24This has prompted this research to explore links between spatial skills and communication abilities, as validated spatial interventions may help improve communication abilities. These current results may also provide unique insights into first-year engineering students’ writing competencies when reporting on a more authentic (non-graded) engineering task. Such information may be useful in eventually shaping guidance of students’ communication instruction in hopes of better preparing them for industry; this is the focus of a planned future research study.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3389/feduc.2020.554806
