skip to main content

Title: Work in Progress – The Development of a K-12 Integrated STEM Observation Protocol.
This WIP presentation is intended to share and gather feedback on the development of an observation protocol for K-12 integrated STEM instruction, the STEM-OP. Specifically, the STEM-OP is being developed for use in K-12 science and/or engineering settings where integrated STEM instruction takes place. While the importance of integrated STEM education is established through national policy documents, there remains disagreement on models and effective approaches for integrated STEM instruction. Our broad definition of integrated STEM includes the use of two or more STEM disciplines to solve a real-world problem or design challenge that supports student development of 21st century skills. This issue is confounded by the lack of observation protocols sensitive to integrated STEM teaching and learning that can be used to inform research of the effectiveness of new models and strategies. Existing instruments most commonly used by researchers, such as the Reformed Teaching Observation Protocol (RTOP), were designed prior to the development of the Next Generation Science Standards and the integration of engineering into science standards. These instruments were also designed for use in reform-based science classrooms, not engineering or integrated STEM learning environments. While engineering-focused observation protocols do exist for K-12 classrooms, they do not evaluate beyond an more » engineering focus, making them limited tools to evaluate integrated STEM instruction. In order to facilitate the implementation of integrated STEM in K-12 classrooms and the development of the nascent integrated STEM education literature, our research team is developing a new integrated STEM observation protocol for use in K-12 science and engineering classrooms. This valid and reliable instrument will be designed for use in a variety of educational contexts and by different education stakeholders to increase the quality of K-12 STEM education. At the end of this project, the STEM-OP will be made available through an online platform that will include an embedded training program to facilitate its broad use. In the first year of this four-year project, we are working on the initial development of the STEM-OP through video analysis and exploratory factor analysis. We are utilizing existing classroom video from a previous project with approximately 2,000 unique classroom videos representing a variety of grade levels (4-9), science content (life, earth, and physical science), engineering design challenges, and school demographics (urban, suburban). The development of the STEM-OP is guided by published frameworks that focus on providing quality K-12 integrated STEM and engineering education, such as the Framework for Quality K-12 Engineering Education. Our anticipated results at the time the ASEE meeting will include a review of our item development process and finalized items included on the draft STEM-OP. Additionally, we anticipate being able to share findings from the exploratory factor analysis (EFA) on our video-coded data, which will identify distinct instructional dimensions responsible for integrated STEM instruction. We value the opportunity to gather feedback from the engineering education community as the integration of engineering design and practices is integral to quality integrated STEM instruction. « less
Authors:
; ; ; ;
Award ID(s):
1812794
Publication Date:
NSF-PAR ID:
10101546
Journal Name:
ASEE Annual Conference proceedings
ISSN:
1524-4644
Sponsoring Org:
National Science Foundation
More Like this
  1. Integrated approaches to teaching science, technology, engineering, and mathematics (commonly referred to as STEM education) in K-12 classrooms have resulted in a growing number of teachers incorporating engineering in their science classrooms. Such changes are a result of shifts in science standards to include engineering as evidenced by the Next Generation Science Standards. To date, 20 states and the District of Columbia have adopted the NGSS and another 24 have adopted standards based on the Framework for K-12 Science Education. Despite the increased presence of engineering and integrated STEM education in K-12 education, there are several concerns to consider. Onemore »concern is the limited availability of observation instruments appropriate for instruction where multiple STEM disciplines are present and integrated with one another. Addressing this concern requires the development of a new observation instrument, designed with integrated STEM instruction in mind. An instrument such as this has implications for both research and practice. For example, research using this instrument could help educators compare integrated STEM instruction across grade bands. Additionally, this tool could be useful in the preparation of pre-service teachers and professional development of in-service teachers new to integrated STEM education and formative learning through professional learning communities or classroom coaching. The work presented here describes in detail the development of an integrated STEM observation instrument - the STEM Observation Protocol (STEM-OP) - that can be used for both research and practice. Over a period of approximately 18-months, a team of STEM educators and educational researchers developed a 10-item integrated STEM observation instrument for use in K-12 science and engineering classrooms. The process of developing the STEM-OP began with establishing a conceptual framework, drawing on the integrated STEM research literature, national standards documents, and frameworks for both K-12 engineering education and integrated STEM education. As part of the instrument development process, the project team had access to over 2000 classroom videos where integrated STEM education took place. Initial analysis of a selection of these videos helped the project team write a preliminary draft instrument consisting of 79 items. Through several rounds of revisions, including the construction of detailed scoring levels of the items and collapsing of items that significantly overlapped, and piloting of the instrument for usability, items were added, edited, and/or removed for various reasons. These reasons included issues concerning the intricacy of the observed phenomenon or the item not being specific to integrated STEM education (e.g., questioning). In its final form, the STEM-OP consists of 10 items, each comprising four descriptive levels. Each item is also accompanied by a set of user guidelines, which have been refined by the project team as a result of piloting the instrument and reviewed by external experts in the field. The instrument has shown to be reliable with the project team and further validation is underway. The STEM-OP will be of use to a wide variety of educators and educational researchers looking to understand the implementation of integrated STEM education in K-12 science and engineering classrooms.« less
  2. Integrated approaches to teaching science, technology, engineering, and mathematics (commonly referred to as STEM education) in K-12 classrooms have resulted in a growing number of teachers incorporating engineering in their science classrooms. Such changes are a result of shifts in science standards to include engineering as evidenced by the Next Generation Science Standards. To date, 20 states and the District of Columbia have adopted the NGSS and another 24 have adopted standards based on the Framework for K-12 Science Education. Despite the increased presence of engineering and integrated STEM education in K-12 education, there are several concerns to consider. Onemore »concern is the limited availability of observation instruments appropriate for instruction where multiple STEM disciplines are present and integrated with one another. Addressing this concern requires the development of a new observation instrument, designed with integrated STEM instruction in mind. An instrument such as this has implications for both research and practice. For example, research using this instrument could help educators compare integrated STEM instruction across grade bands. Additionally, this tool could be useful in the preparation of pre-service teachers and professional development of in-service teachers new to integrated STEM education and formative learning through professional learning communities or classroom coaching. The work presented here describes in detail the development of an integrated STEM observation instrument that can be used for both research and practice. Over a period of approximately 18-months, a team of STEM educators and educational researchers developed a 10-item integrated STEM observation instrument for use in K-12 science and engineering classrooms. The process of developing the instrument began with establishing a conceptual framework, drawing on the integrated STEM research literature, national standards documents, and frameworks for both K-12 engineering education and integrated STEM education. As part of the instrument development process, the project team had access to over 2000 classroom videos where integrated STEM education took place. Initial analysis of a selection of these videos helped the project team write a preliminary draft instrument consisting of 52 items. Through several rounds of revisions, including the construction of detailed scoring levels of the items and collapsing of items that significantly overlapped, and piloting of the instrument for usability, items were added, edited, and/or removed for various reasons. These reasons included issues concerning the intricacy of the observed phenomenon or the item not being specific to integrated STEM education (e.g., questioning). In its final form, the instrument consists of 10 items, each comprising four descriptive levels. Each item is also accompanied by a set of user guidelines, which have been refined by the project team as a result of piloting the instrument and reviewed by external experts in the field. The instrument has shown to be reliable with the project team and further validation is underway. This instrument will be of use to a wide variety of educators and educational researchers looking to understand the implementation of integrated STEM education in K-12 science and engineering classrooms.« less
  3. Engineering Explorations are curriculum modules that engage children across contexts in learning about science and engineering. We used them to leverage multiple education sectors (K–12 schools, museums, higher education, and afterschool programs) across a community to provide engineering learning experiences for youth, while increasing local teachers’ capacity to deliver high-quality engineering learning opportunities that align with school standards. Focusing on multiple partners that serve youth in the same community provides opportunities for long-term collaborations and programs developed in response to local needs. In a significant shift from earlier sets of standards, the Next Generation Science Standards include engineering design, withmore »the goal of providing students with a foundation “to better engage in and aspire to solve the major societal and environmental challenges they will face in decades ahead” (NGSS Lead States 2013, Appendix I). Including engineering in K–12 standards is a positive step forward in introducing students to engineering; however, K–12 teachers are not prepared to facilitate high-quality engineering activities. Research has consistently shown that elementary teachers are not confident in teaching science, especially physical science, and generally have little knowledge of engineering (Trygstad 2013). K–12 teachers, therefore, will need support. Our goal was to create a program that took advantage of the varied resources across a STEM (science, technology, engineering, and math) education ecosystem to support engineering instruction for youth across multiple contexts, while building the capacity of educators and meeting the needs of each organization. Specifically, we developed mutually reinforcing classroom and field trip activities to improve student learning and a curriculum to improve teacher learning. This challenging task required expertise in school-based standards, engineering education, informal education, teacher professional development, and classroom and museum contexts.« less
  4. Flexible classroom spaces, which have movable tables and chairs that can be easily rearranged into different layouts, make it easier for instructors to effectively implement active learning than a traditional lecture hall. Instructors can move throughout the room to interact with students during active learning, and they can rearrange the tables into small groups to facilitate conversation between students. Classroom technology, such as wall-mounted monitors and movable whiteboards, also facilitates active learning by allowing students to collaborate. In addition to enabling active learning, the flexible classroom can still be arranged in front-facing rows that support traditional lecture-based pedagogies. As amore »result, instructors do not have to make time- and effort-intensive changes to the way their courses are taught in order to use the flexible classroom. Instead, they can make small changes to add active learning. We are in the second year of a study of flexible classroom spaces funded by the National Science Foundation’s Division of Undergraduate Education. This project asks four research questions that investigate the relationships between the instructor, the students, and the classroom: 1) What pedagogy do instructors use in a flexible classroom space? 2) How do instructors take advantage of the instructional affordances (including the movable furniture, movable whiteboards, wall-mounted whiteboards, and wall-mounted monitors) of a flexible classroom? 3) What is the impact of faculty professional development on instructors’ use of flexible classroom spaces? and 4) How does the classroom influence the ways students interpret and engage in group learning activities? In the first year of our study we have developed five research instruments to answer these questions: a three-part classroom observation protocol, an instructor interview protocol, two instructor surveys, and a student survey. We have collected data from nine courses taught in one of ten flexible classrooms at the University of Michigan during the Fall 2018 semester. Two of these courses were first-year introduction to engineering courses co-taught by two instructors, and the other seven courses were sophomore- and junior-level core technical courses taught by one instructor. Five instructors participated in a faculty learning community that met three times during the semester to discuss active learning, to learn how to make the best use of the flexible classroom affordances, and to plan activities to implement in their courses. In each course we gathered data from the perspective of the instructor (through pre- and post-semester interviews), the researcher (through observations of three class meetings with our observation protocol), and the students (through conducting a student survey at the end of the semester). This poster presents qualitative and qualitative analyses of these data to answer our research questions, along with evidence based best practices for effectively using a flexible classroom.« less
  5. Communication of ideas involves the simultaneous efforts of verbal, physical and neurological processes (Sherr, 2008). In elementary classrooms where young students are in the process of developing their verbal capacities, gestures from both the teacher and students serve as a key component of communication of new ideas and the processing of social information (Foglia & Wilson, 2013). Thus far, research efforts to understand how students utilize gestures in the communication and understanding of ideas have focused primarily on mathematics and the physical sciences (see Nemirovsky & Ferrara, 2009; Nuñez, Edwards & Matos, 1999; Shapiro, 2014; Sherr, 2008). With the introductionmore »of the Next Generation Science Standards (NGSS Lead States, 2013), students engineering is now included in K-12 instruction. Engineering education centers around designing and optimizing solutions to engineering challenges. The creation of a design solution differentiates engineering education from other classroom subject areas. Current work in engineering education focuses mostly on students’ words or drawings, leaving out gestures as an important component of students' communication of engineering designs. This study aimed to contribute to the general understanding of students’ use of gestures and manipulatives when discussing their engineering design solutions and is part of a larger NSF-funded project. Students participated in pre- and post-field trip classroom activities that extended learning done on an engineering-focused field trip to the local science center into the classroom. For this study, we focused on a module that challenged students to design a craft that either slowed the fall of a penny (classroom engineering design challenge) or hovered in a column of upward moving air (field trip engineering design challenge). We analyzed six videos (3 from the classroom and 3 from the field trip) of first-grade student explanations of their crafts to identify their use of gestures and prototyped craft design solutions in communicating. In this paper, we explore how student use of gestures and use of prototyped design solutions overlap and differentiate to understand how student sense-making can be understood through each.« less