More Like this

1. Work in Progress – The Development of a K-12 Integrated STEM Observation Protocol.

   Roehrig, G.H ; Ring-Whalen, E.A. ; Wieselmann, J.R. ; Dare, E.A. ; Ellis, J.A. ( January 2019 , ASEE Annual Conference proceedings)

   This WIP presentation is intended to share and gather feedback on the development of an observation protocol for K-12 integrated STEM instruction, the STEM-OP. Specifically, the STEM-OP is being developed for use in K-12 science and/or engineering settings where integrated STEM instruction takes place. While the importance of integrated STEM education is established through national policy documents, there remains disagreement on models and effective approaches for integrated STEM instruction. Our broad definition of integrated STEM includes the use of two or more STEM disciplines to solve a real-world problem or design challenge that supports student development of 21st century skills. This issue is confounded by the lack of observation protocols sensitive to integrated STEM teaching and learning that can be used to inform research of the effectiveness of new models and strategies. Existing instruments most commonly used by researchers, such as the Reformed Teaching Observation Protocol (RTOP), were designed prior to the development of the Next Generation Science Standards and the integration of engineering into science standards. These instruments were also designed for use in reform-based science classrooms, not engineering or integrated STEM learning environments. While engineering-focused observation protocols do exist for K-12 classrooms, they do not evaluate beyond anmore »engineering focus, making them limited tools to evaluate integrated STEM instruction. In order to facilitate the implementation of integrated STEM in K-12 classrooms and the development of the nascent integrated STEM education literature, our research team is developing a new integrated STEM observation protocol for use in K-12 science and engineering classrooms. This valid and reliable instrument will be designed for use in a variety of educational contexts and by different education stakeholders to increase the quality of K-12 STEM education. At the end of this project, the STEM-OP will be made available through an online platform that will include an embedded training program to facilitate its broad use. In the first year of this four-year project, we are working on the initial development of the STEM-OP through video analysis and exploratory factor analysis. We are utilizing existing classroom video from a previous project with approximately 2,000 unique classroom videos representing a variety of grade levels (4-9), science content (life, earth, and physical science), engineering design challenges, and school demographics (urban, suburban). The development of the STEM-OP is guided by published frameworks that focus on providing quality K-12 integrated STEM and engineering education, such as the Framework for Quality K-12 Engineering Education. Our anticipated results at the time the ASEE meeting will include a review of our item development process and finalized items included on the draft STEM-OP. Additionally, we anticipate being able to share findings from the exploratory factor analysis (EFA) on our video-coded data, which will identify distinct instructional dimensions responsible for integrated STEM instruction. We value the opportunity to gather feedback from the engineering education community as the integration of engineering design and practices is integral to quality integrated STEM instruction.« less
2. Improving Integrated STEM Education: The Design and Development of a K-12 STEM Observation Protocol (STEM-OP) (RTP)

   Dare, E. A. ; Hiwatig, B. ; Keratithamkul, K. ; Ellis, J. A. ; Roehrig, G. H. ; Ring-Whalen, E. A. ; Rouleau, M. D. ; Faruqi, F. ; Rice, C. ; Titu, P. ; et al ( January 2021 , ASEE Annual Conference proceedings)

   Integrated approaches to teaching science, technology, engineering, and mathematics (commonly referred to as STEM education) in K-12 classrooms have resulted in a growing number of teachers incorporating engineering in their science classrooms. Such changes are a result of shifts in science standards to include engineering as evidenced by the Next Generation Science Standards. To date, 20 states and the District of Columbia have adopted the NGSS and another 24 have adopted standards based on the Framework for K-12 Science Education. Despite the increased presence of engineering and integrated STEM education in K-12 education, there are several concerns to consider. One concern is the limited availability of observation instruments appropriate for instruction where multiple STEM disciplines are present and integrated with one another. Addressing this concern requires the development of a new observation instrument, designed with integrated STEM instruction in mind. An instrument such as this has implications for both research and practice. For example, research using this instrument could help educators compare integrated STEM instruction across grade bands. Additionally, this tool could be useful in the preparation of pre-service teachers and professional development of in-service teachers new to integrated STEM education and formative learning through professional learning communities or classroommore »coaching. The work presented here describes in detail the development of an integrated STEM observation instrument - the STEM Observation Protocol (STEM-OP) - that can be used for both research and practice. Over a period of approximately 18-months, a team of STEM educators and educational researchers developed a 10-item integrated STEM observation instrument for use in K-12 science and engineering classrooms. The process of developing the STEM-OP began with establishing a conceptual framework, drawing on the integrated STEM research literature, national standards documents, and frameworks for both K-12 engineering education and integrated STEM education. As part of the instrument development process, the project team had access to over 2000 classroom videos where integrated STEM education took place. Initial analysis of a selection of these videos helped the project team write a preliminary draft instrument consisting of 79 items. Through several rounds of revisions, including the construction of detailed scoring levels of the items and collapsing of items that significantly overlapped, and piloting of the instrument for usability, items were added, edited, and/or removed for various reasons. These reasons included issues concerning the intricacy of the observed phenomenon or the item not being specific to integrated STEM education (e.g., questioning). In its final form, the STEM-OP consists of 10 items, each comprising four descriptive levels. Each item is also accompanied by a set of user guidelines, which have been refined by the project team as a result of piloting the instrument and reviewed by external experts in the field. The instrument has shown to be reliable with the project team and further validation is underway. The STEM-OP will be of use to a wide variety of educators and educational researchers looking to understand the implementation of integrated STEM education in K-12 science and engineering classrooms.« less
3. Improving STEM education: The design and development of a K-12 STEM observation protocol (STEM-OP).

   Dare, E.A. ; Hiwatig, B. ; Karatithamkul, K. ; Ellis, J.A. ; Roehrig, G.H. ; Ring-Whalen, E.A. ; Rouleau, M.D. ; Faruqi, F. ; Rice, C. ; Titu, P. ; et al ( January 2021 , ASEE Annual Conference proceedings)

   Integrated approaches to teaching science, technology, engineering, and mathematics (commonly referred to as STEM education) in K-12 classrooms have resulted in a growing number of teachers incorporating engineering in their science classrooms. Such changes are a result of shifts in science standards to include engineering as evidenced by the Next Generation Science Standards. To date, 20 states and the District of Columbia have adopted the NGSS and another 24 have adopted standards based on the Framework for K-12 Science Education. Despite the increased presence of engineering and integrated STEM education in K-12 education, there are several concerns to consider. One concern is the limited availability of observation instruments appropriate for instruction where multiple STEM disciplines are present and integrated with one another. Addressing this concern requires the development of a new observation instrument, designed with integrated STEM instruction in mind. An instrument such as this has implications for both research and practice. For example, research using this instrument could help educators compare integrated STEM instruction across grade bands. Additionally, this tool could be useful in the preparation of pre-service teachers and professional development of in-service teachers new to integrated STEM education and formative learning through professional learning communities or classroommore »coaching. The work presented here describes in detail the development of an integrated STEM observation instrument that can be used for both research and practice. Over a period of approximately 18-months, a team of STEM educators and educational researchers developed a 10-item integrated STEM observation instrument for use in K-12 science and engineering classrooms. The process of developing the instrument began with establishing a conceptual framework, drawing on the integrated STEM research literature, national standards documents, and frameworks for both K-12 engineering education and integrated STEM education. As part of the instrument development process, the project team had access to over 2000 classroom videos where integrated STEM education took place. Initial analysis of a selection of these videos helped the project team write a preliminary draft instrument consisting of 52 items. Through several rounds of revisions, including the construction of detailed scoring levels of the items and collapsing of items that significantly overlapped, and piloting of the instrument for usability, items were added, edited, and/or removed for various reasons. These reasons included issues concerning the intricacy of the observed phenomenon or the item not being specific to integrated STEM education (e.g., questioning). In its final form, the instrument consists of 10 items, each comprising four descriptive levels. Each item is also accompanied by a set of user guidelines, which have been refined by the project team as a result of piloting the instrument and reviewed by external experts in the field. The instrument has shown to be reliable with the project team and further validation is underway. This instrument will be of use to a wide variety of educators and educational researchers looking to understand the implementation of integrated STEM education in K-12 science and engineering classrooms.« less
4. Studying In-service Teacher Professional Development on Purposeful Integration of Engineering into K-12 STEM Teaching (Research to Practice)

   Gunning, A. M. ( July 2021 , American Society for Engineering Education 2021 Annual Conference)

   Integrated STEM approaches in K-12 science and math instruction can be more engaging and meaningful for students and often meet the curriculum content and practice goals better than single-subject lessons. Engineering, as a key component of STEM education, offers hands-on, designed-based, problem solving activities to drive student interest and confidence in STEM overall. However, K-12 STEM teachers may not feel equipped to implement engineering practices and may even experience anxiety about trying them out in their classrooms without the added support of professional development and professional learning communities. To address these concerns and support engineering integration, this research study examined the experiences of 18 teachers in one professional development program dedicated to STEM integration and engineering pedagogy for K-12 classrooms. This professional development program positioned the importance of the inclusion of engineering content and encouraged teachers to explore community-based, collaborative activities that identified and spoke to societal needs and social impacts through engineering integration. Data collected from two of the courses in this project, Enhancing Mathematics with STEM and Engineering in the K-12 Classroom, included participant reflections, focus groups, microteaching lesson plans, and field notes. Through a case study approach and grounded theory analysis, themes of self-efficacy, active learning supports,more »and social justice teaching emerged. The following discussion on teachers’ engineering and STEM self-efficacy, teachers’ integration of engineering to address societal needs and social impacts, and teachers’ development in engineering education through hands-on activities, provides better understanding of engineering education professional development for K-12 STEM teachers.« less
5. Long-Term Outcomes of RET Programs on Female and Minority Student High School Graduation Rates and Undergraduate STEM Major Rates (Fundamental)

   Nichol, C. ; Crawford, C. A. ; Barr, C. ; Cerda, I. ( January 2021 , ASEE Annual Conference proceedings)

   Research Experience for Teachers (RET) programs are National Science Foundation (NSF) funded programs designed to provide K- 12 Science, Technology, Engineering, and Mathematics (STEM) teachers with immersive, hands-on research experiences at Universities around the country. The NSF RET in nanotechnology encourages teachers to translate cutting-edge research into culturally relevant Project-Based Learning (PjBL) and engineering curriculum. Traditionally, the evaluation of RET programs focuses on the growth and development of teacher self-efficacy, engineering content knowledge gains, or classroom implementation of developed curriculum materials. However, reported methods for evaluating the impact of RETs on their female, minority student populations' high school graduation and undergraduate STEM major rates are limited. This study's objective was to compare RET high school student graduation rates and undergraduate STEM major rates across gender, race, and ethnicity to a comparison sample to determine the RET program's long-term impact on students' likelihood of pursuing STEM careers. The approach of collecting and analyzing the Texas Education Research Center Database (EdRC) data is a novel methodology for assessing RET programs' effectiveness on students. The EdRC is a repository of K-12 student data from the Texas Education Agency (TEA) and Higher Education data from the Texas Higher Education Coordinating Board (THECB). This jointmore »database contains demographic, course registration, graduation, standardized testing, and college major, among others, for all students that attended a K-12 public school in Texas and any college in Texas, public or private. The RET program participants at Rice University (2010 – 2018) taught numerous students, a sample size of 11,240 students. A propensity score matching generated the student comparison group within the database. Students' school campus, gender, race/ethnic status, and English proficiency status were applied to produce a graduation comparison sample size of 11,240 students of Non-RET participants. Linking the TEA database to the THECB database resulted in college STEM participants and comparison sample sizes of 4,029 students. The project team conducted a logistic regression using RET status to predict high school graduation rates as a whole and by individual variables: gender, Asian American, Black, Caucasian, and Latinx students. All models were significant at p less than 0.05, with models in favor of students RET teachers. The project team conducted a logistic regression using RET status to predict student STEM undergraduate major rates as a whole and by individual variables: Gender, Asian American, Black, Caucasian, and Latinx students. African American and Caucasian models were significant at p less than 0.05; Gender, Asian American, and Latinx models were marginally significant (0.05 less than p greater than 0.1), where RET students had higher STEM major rates than matched controls. The findings demonstrate that RET programs have a long-term positive impact on the students' high school graduation rates and undergraduate STEM major rates. As teachers who participate in the RET programs are more likely to conduct courses using PjBL strategies and incorporate real-world engineering practices, female and minority students are more likely to benefit from these practices and seek careers utilizing these skills.« less