More Like this

1. Work in Progress – The Development of a K-12 Integrated STEM Observation Protocol.

   Roehrig, G.H ; Ring-Whalen, E.A. ; Wieselmann, J.R. ; Dare, E.A. ; Ellis, J.A. ( January 2019 , ASEE Annual Conference proceedings)

   This WIP presentation is intended to share and gather feedback on the development of an observation protocol for K-12 integrated STEM instruction, the STEM-OP. Specifically, the STEM-OP is being developed for use in K-12 science and/or engineering settings where integrated STEM instruction takes place. While the importance of integrated STEM education is established through national policy documents, there remains disagreement on models and effective approaches for integrated STEM instruction. Our broad definition of integrated STEM includes the use of two or more STEM disciplines to solve a real-world problem or design challenge that supports student development of 21st century skills. This issue is confounded by the lack of observation protocols sensitive to integrated STEM teaching and learning that can be used to inform research of the effectiveness of new models and strategies. Existing instruments most commonly used by researchers, such as the Reformed Teaching Observation Protocol (RTOP), were designed prior to the development of the Next Generation Science Standards and the integration of engineering into science standards. These instruments were also designed for use in reform-based science classrooms, not engineering or integrated STEM learning environments. While engineering-focused observation protocols do exist for K-12 classrooms, they do not evaluate beyond anmore »engineering focus, making them limited tools to evaluate integrated STEM instruction. In order to facilitate the implementation of integrated STEM in K-12 classrooms and the development of the nascent integrated STEM education literature, our research team is developing a new integrated STEM observation protocol for use in K-12 science and engineering classrooms. This valid and reliable instrument will be designed for use in a variety of educational contexts and by different education stakeholders to increase the quality of K-12 STEM education. At the end of this project, the STEM-OP will be made available through an online platform that will include an embedded training program to facilitate its broad use. In the first year of this four-year project, we are working on the initial development of the STEM-OP through video analysis and exploratory factor analysis. We are utilizing existing classroom video from a previous project with approximately 2,000 unique classroom videos representing a variety of grade levels (4-9), science content (life, earth, and physical science), engineering design challenges, and school demographics (urban, suburban). The development of the STEM-OP is guided by published frameworks that focus on providing quality K-12 integrated STEM and engineering education, such as the Framework for Quality K-12 Engineering Education. Our anticipated results at the time the ASEE meeting will include a review of our item development process and finalized items included on the draft STEM-OP. Additionally, we anticipate being able to share findings from the exploratory factor analysis (EFA) on our video-coded data, which will identify distinct instructional dimensions responsible for integrated STEM instruction. We value the opportunity to gather feedback from the engineering education community as the integration of engineering design and practices is integral to quality integrated STEM instruction.« less
2. Improving Integrated STEM Education: The Design and Development of a K-12 STEM Observation Protocol (STEM-OP) (RTP)

   Dare, E. A. ; Hiwatig, B. ; Keratithamkul, K. ; Ellis, J. A. ; Roehrig, G. H. ; Ring-Whalen, E. A. ; Rouleau, M. D. ; Faruqi, F. ; Rice, C. ; Titu, P. ; et al ( January 2021 , ASEE Annual Conference proceedings)

   Integrated approaches to teaching science, technology, engineering, and mathematics (commonly referred to as STEM education) in K-12 classrooms have resulted in a growing number of teachers incorporating engineering in their science classrooms. Such changes are a result of shifts in science standards to include engineering as evidenced by the Next Generation Science Standards. To date, 20 states and the District of Columbia have adopted the NGSS and another 24 have adopted standards based on the Framework for K-12 Science Education. Despite the increased presence of engineering and integrated STEM education in K-12 education, there are several concerns to consider. One concern is the limited availability of observation instruments appropriate for instruction where multiple STEM disciplines are present and integrated with one another. Addressing this concern requires the development of a new observation instrument, designed with integrated STEM instruction in mind. An instrument such as this has implications for both research and practice. For example, research using this instrument could help educators compare integrated STEM instruction across grade bands. Additionally, this tool could be useful in the preparation of pre-service teachers and professional development of in-service teachers new to integrated STEM education and formative learning through professional learning communities or classroommore »coaching. The work presented here describes in detail the development of an integrated STEM observation instrument - the STEM Observation Protocol (STEM-OP) - that can be used for both research and practice. Over a period of approximately 18-months, a team of STEM educators and educational researchers developed a 10-item integrated STEM observation instrument for use in K-12 science and engineering classrooms. The process of developing the STEM-OP began with establishing a conceptual framework, drawing on the integrated STEM research literature, national standards documents, and frameworks for both K-12 engineering education and integrated STEM education. As part of the instrument development process, the project team had access to over 2000 classroom videos where integrated STEM education took place. Initial analysis of a selection of these videos helped the project team write a preliminary draft instrument consisting of 79 items. Through several rounds of revisions, including the construction of detailed scoring levels of the items and collapsing of items that significantly overlapped, and piloting of the instrument for usability, items were added, edited, and/or removed for various reasons. These reasons included issues concerning the intricacy of the observed phenomenon or the item not being specific to integrated STEM education (e.g., questioning). In its final form, the STEM-OP consists of 10 items, each comprising four descriptive levels. Each item is also accompanied by a set of user guidelines, which have been refined by the project team as a result of piloting the instrument and reviewed by external experts in the field. The instrument has shown to be reliable with the project team and further validation is underway. The STEM-OP will be of use to a wide variety of educators and educational researchers looking to understand the implementation of integrated STEM education in K-12 science and engineering classrooms.« less
3. Improving STEM education: The design and development of a K-12 STEM observation protocol (STEM-OP).

   Dare, E.A. ; Hiwatig, B. ; Karatithamkul, K. ; Ellis, J.A. ; Roehrig, G.H. ; Ring-Whalen, E.A. ; Rouleau, M.D. ; Faruqi, F. ; Rice, C. ; Titu, P. ; et al ( January 2021 , ASEE Annual Conference proceedings)

   Integrated approaches to teaching science, technology, engineering, and mathematics (commonly referred to as STEM education) in K-12 classrooms have resulted in a growing number of teachers incorporating engineering in their science classrooms. Such changes are a result of shifts in science standards to include engineering as evidenced by the Next Generation Science Standards. To date, 20 states and the District of Columbia have adopted the NGSS and another 24 have adopted standards based on the Framework for K-12 Science Education. Despite the increased presence of engineering and integrated STEM education in K-12 education, there are several concerns to consider. One concern is the limited availability of observation instruments appropriate for instruction where multiple STEM disciplines are present and integrated with one another. Addressing this concern requires the development of a new observation instrument, designed with integrated STEM instruction in mind. An instrument such as this has implications for both research and practice. For example, research using this instrument could help educators compare integrated STEM instruction across grade bands. Additionally, this tool could be useful in the preparation of pre-service teachers and professional development of in-service teachers new to integrated STEM education and formative learning through professional learning communities or classroommore »coaching. The work presented here describes in detail the development of an integrated STEM observation instrument that can be used for both research and practice. Over a period of approximately 18-months, a team of STEM educators and educational researchers developed a 10-item integrated STEM observation instrument for use in K-12 science and engineering classrooms. The process of developing the instrument began with establishing a conceptual framework, drawing on the integrated STEM research literature, national standards documents, and frameworks for both K-12 engineering education and integrated STEM education. As part of the instrument development process, the project team had access to over 2000 classroom videos where integrated STEM education took place. Initial analysis of a selection of these videos helped the project team write a preliminary draft instrument consisting of 52 items. Through several rounds of revisions, including the construction of detailed scoring levels of the items and collapsing of items that significantly overlapped, and piloting of the instrument for usability, items were added, edited, and/or removed for various reasons. These reasons included issues concerning the intricacy of the observed phenomenon or the item not being specific to integrated STEM education (e.g., questioning). In its final form, the instrument consists of 10 items, each comprising four descriptive levels. Each item is also accompanied by a set of user guidelines, which have been refined by the project team as a result of piloting the instrument and reviewed by external experts in the field. The instrument has shown to be reliable with the project team and further validation is underway. This instrument will be of use to a wide variety of educators and educational researchers looking to understand the implementation of integrated STEM education in K-12 science and engineering classrooms.« less
4. Examining the Literacy Practices of Engineers to Develop a Model of Disciplinary Literacy Instruction for K-12 Engineering (Work in Progress)

   Green, Theresa ; Minichiello, Angela ; Wilson-Lopez, Amy ( June 2018 , ASEE Annual Conference proceedings)

   Despite efforts to diversify the science, technology, engineering, and mathematics (STEM) workforce, engineering remains a White, male-dominated profession. Often, women and underrepresented students do not identify with STEM careers and many opt out of STEM pathways prior to entering high school or college. In order to broaden participation in engineering, new methods of engaging and retaining those who are traditionally underrepresented in engineering are needed. This work is based on a promising approach for encouraging and supporting diverse participation in engineering: disciplinary literacy instruction (DLI). Generally, teachers use DLI to provide K-12 students with a framework for interpreting, evaluating, and generating discipline-specific texts. This instruction provides students with an understanding of how experts in the discipline read, engage, and generate texts used to solve problems or communicate information. While models of disciplinary literacy have been developed and disseminated in several humanities and science fields, there is a lack of empirical and theoretical research that examines the use of DLI within the engineering domain. It is thought that DLI can be used to foster diverse student interest in engineering from a young age by removing literacy-based barriers that often discourage underrepresented students from entering and pursuing careers in STEM fields. Thismore »work-in-progress paper describes a new study underway to develop and disseminate a model of disciplinary literacy in engineering. During this project, researchers will observe, interview, and collect written artifacts from engineers working across four sub-disciplines of engineering: aerospace/mechanical, biological, civil/environmental, and electrical/computer. Data that will be collected include interview transcripts, observation field notes, engineer logs of literacy practices, and photographs of texts that the engineers read and write. Data will be analyzed using constant comparative analytic (CCA) methods. CCA will be used to generate theoretical codes from the data that will form the basis for a model of disciplinary literacy in engineering. As a primary outcome of this research, the engineering DLI model will promote the use of DLI practices within K-12 engineering instruction in order to assist and encourage diverse, underrepresented students to engage in engineering courses of study and pursue STEM careers. Thus far, the research team has begun collecting and analyzing data from two electrical engineers. This work in progress paper will report on preliminary findings, as well as implications for K-12 classroom instruction. For instance, this study has shed insights on how engineers use texts as part of the process of conducting failure analysis, and the research team has begun to conceptualize how these types of texts might be used with K-12 students to help them conduct failure analyses during design testing. Ultimately, this project will result in a list of grade-appropriate texts, evaluative frameworks, and activities (e.g., failure analysis in testing) that K-12 engineering teachers can use to prepare their diverse students to think, act, read, and write like engineers.« less
5. Analyzing K-12 Education as a Complex System

   Llewellyn, D. C. ( July 2013 , ASEE annual conference & exposition)

   Schools and school districts are complex, dynamic systems affected by numerous factors, specific to the particular environment. These factors, which range from the stability of the home life of the enrolled children, to the interpersonal relationships of the school staff, to the funding decisions of the school board, to the laws passed by the U.S. Congress (and innumerable additional factors in between), all interact in sometimes predictable but often completely surprising ways. Educational initiatives and interventions that work well in one environment can prove completely ineffective (or un-implementable) in a different school setting, for a myriad of reasons. For university faculty and STEM professionals who partner with K-12 schools to implement and assess STEM educational reform initiatives, particularly for those who choose to work or scale up projects in non-charter or non-specialized lab school settings, the complexity of the system of K-12 education makes it difficult to identify all the potential barriers that can impact the proposed project. Unexpected factors can easily derail an otherwise well thought-out project, both in terms of project implementation and also in the success of assessing student outcomes. Educational researchers have long studied school reform and the issues of what facilitates and hinders success inmore »curricular and other interventions. Experts in educational policy and public policy also have studied the interaction of policies and practices of reform agendas within social and organizational contexts. Industrial engineering, which had its origins in studying manufacturing systems, is a field where researchers have made great contributions towards understanding complex systems including transportation systems, financial systems, health care, and even recently humanitarian support systems. The Advanced Manufacturing and Prototyping Integrated to Unlock Potential (AMP-IT-UP) NSF Math/Science Partnership at the Georgia Institute of Technology is creating an innovative framework, which is both conceptual and theoretical and rooted within the field of industrial and systems engineering, to examine barriers and enablers to school change and reform. The framework describes the system in terms of both agents and the attributes of those agents and will become the foundation for identifying a subset of attribute combinations that allow for successful change in the system. In this paper we describe the first step in creating this framework, namely identifying the agents within K-12 education and the attributes of these agents that are critical to educational change. The paper also presents a sample scale for describing these attributes.« less