More Like this

1. Developing a measure to capture middle school students’ interpretive understanding of engineering design

   Piña, Jeremiah ; Ellis, Glenn W ; Rudnitsky, Al ; Mazur, Rebecca ; McGinnis-Cavanaugh, Beth ; Huff, Isabel ( July 2021 , 2021 American Society for Engineering Education Annual Conference)

   null (Ed.)

   This research paper describes the development of an assessment instrument for use with middle school students that provides insight into students’ interpretive understanding by looking at early indicators of developing expertise in students’ responses to solution generation, reflection, and concept demonstration tasks. We begin by detailing a synthetic assessment model that served as the theoretical basis for assessing specific thinking skills. We then describe our process of developing test items by working with a Teacher Design Team (TDT) of instructors in our partner school system to set guidelines that would better orient the assessment in that context and working within the framework of standards and disciplinary core ideas enumerated in the Next Generation Science Standards (NGSS). We next specify our process of refining the assessment from 17 items across three separate item pools to a final total of three open-response items. We then provide evidence for the validity and reliability of the assessment instrument from the standards of (1) content, (2) meaningfulness, (3) generalizability, and (4) instructional sensitivity. As part of the discussion from the standards of generalizability and instructional sensitivity, we detail a study carried out in our partner school system in the fall of 2019. The instrument was administered to students in treatment (n= 201) and non- treatment (n = 246) groups, wherein the former participated in a two-to-three- week, NGSS-aligned experimental instructional unit introducing the principles of engineering design that focused on engaging students using the Imaginative Education teaching approach. The latter group were taught using the district’s existing engineering design curriculum. Results from statistical analysis of student responses showed that the interrater reliability of the scoring procedures were good-to-excellent, with intra-class correlation coefficients ranging between .72 and .95. To gauge the instructional sensitivity of the assessment instrument, a series of non-parametric comparative analyses (independent two-group Mann- Whitney tests) were carried out. These found statistically significant differences between treatment and non-treatment student responses related to the outcomes of fluency and elaboration, but not reflection. 

   more » « less
2. Evaluation of Engineering Problem-framing Professional Development for K-12 Science Teachers

   https://doi.org/10.18260/1-2--34604  

   West, Meg E ; Hylton, J Blake ; Herak, Patrick J ; Wellman, Bruce ; France, Todd ( June 2020 , 2020 ASEE Virtual Annual Conference)

   As the importance to integrate engineering into K12 curricula grows so does the need to develop teachers’ engineering teaching capabilities and knowledge. One method that has been used to aid this development is engineering professional development programs. This evaluation paper presents the successes and challenges of an engineering professional development program for teachers focused around the use of engineering problem-framing design activities in high school science classrooms. These activities were designed to incorporate the cross-cutting ideas published in the Next Generation Science Standards (NGSS) and draw on best practices for instructional design of problem-framing activities from research on design and model-eliciting activities (MEAs). The professional development (PD) was designed to include the following researched-based effective PD key elements: (1) is content focused, (2) incorporates active learning, (3) supports collaboration, (4) uses models of effective practice, (5) provides coaching and expert support, (6) offers feedback and reflection, and (7) is of sustained duration. The engineering PD, including in-classroom deployment of activities and data collection, was designed as an iterative process to be conducted over a three-year period. This will allow for improvement and refinement of our approach. The first iteration, reported in this paper, consisted of seven high school science teachers who have agreed to participate in the PD, implement the problem-framing activities, and collect student data over a period of one year. The PD itself consisted of the teachers comparing science and engineering, participating in problem-framing training and activities, and developing a design challenge scenario for their own courses. The participating teachers completed a survey at the end of the PD that will be used to inform enhancement of the PD and our efforts to recruit additional participants in the following year. The qualitative survey consisted of open-ended questions asking for the most valuable takeaways from the PD, their reasoning for joining the PD, reasons they would or would not recommend the PD, and, in their opinion, what would inspire their colleagues to attend the PD. The responses to the survey along with observations from the team presenting the PD were analyzed to identify lessons learned and future steps for the following iteration of the PD. From the data, three themes emerged: Development of PD, Teacher Motivation, and Teacher Experience. 

   more » « less
3. The First Year of an Undergraduate Service Learning Partnership to Enhance Engineering Education and Elementary Pre-Service Teacher Education

   https://doi.org/10.18260/1-2--35320  

   Ringleb, S. ; Kidd, J. ; Pazos, P. ; Gutierrez, K. ; Ayala, O. ; Kaipa, K. ( June 2020 , 2020 ASEE Virtual Annual Conference Content Access, Virtual Online)

   This project was designed to address three major challenges faced by undergraduate engineering students (UES) and pre-service teachers (PSTs): 1) retention for UESs after the first year, and continued engagement when they reach more difficult concepts, 2) to prepare PSTs to teach engineering, which is a requirement in the Next Generation Science Standards as well as many state level standards of learning, and 3) to prepare both groups of students to communicate and collaborate in a multi-disciplinary context, which is a necessary skill in their future places of work. This project was implemented in three pairs of classes: 1) an introductory mechanical engineering class, fulfilling a general education requirement for information literacy and a foundations class in education, 2) fluid mechanics in mechanical engineering technology and a science methods class in education, and 3) mechanical engineering courses requiring programming (e.g., computational methods and robotics) with an educational technology class. All collaborations taught elementary level students (4th or 5th grade). For collaborations 1 and 2, the elementary students came to campus for a field trip where they toured engineering labs and participated in a one hour lesson taught by both the UESs and PSTs. In collaboration 3, the UESs and PSTs worked with the upper-elementary students in their school during an after school club. In collaborations 1 and 2, students were assigned to teams and worked remotely on some parts of the project. A collaboration tool, built in Google Sites and Google Drive, was used to facilitate the project completion. The collaboration tool includes a team repository for all the project documents and templates. Students in collaboration 3 worked together directly during class time on smaller assignments. In all three collaborations lesson plans were implemented using the BSCS 5E instructional model, which was aligned to the engineering design process. Instruments were developed to assess knowledge in collaborations 1 (engineering design process) and 3 (computational thinking), while in collaboration 2, knowledge was assessed with questions from the fundamentals of engineering exam and a science content assessment. Comprehensive Assessment of Team Member Effectiveness (CATME) was also used in all 3 collaborations to assess teamwork across the collaborations. Finally, each student wrote a reflection on their experiences, which was used to qualitatively assess the project impact. The results from the first full semester of implementation have led us to improvements in the implementation and instrument refinement for year 2. 

   more » « less
4. Work in Progress – The Development of a K-12 Integrated STEM Observation Protocol.

   Roehrig, G.H ; Ring-Whalen, E.A. ; Wieselmann, J.R. ; Dare, E.A. ; Ellis, J.A. ( January 2019 , ASEE Annual Conference proceedings)

   This WIP presentation is intended to share and gather feedback on the development of an observation protocol for K-12 integrated STEM instruction, the STEM-OP. Specifically, the STEM-OP is being developed for use in K-12 science and/or engineering settings where integrated STEM instruction takes place. While the importance of integrated STEM education is established through national policy documents, there remains disagreement on models and effective approaches for integrated STEM instruction. Our broad definition of integrated STEM includes the use of two or more STEM disciplines to solve a real-world problem or design challenge that supports student development of 21st century skills. This issue is confounded by the lack of observation protocols sensitive to integrated STEM teaching and learning that can be used to inform research of the effectiveness of new models and strategies. Existing instruments most commonly used by researchers, such as the Reformed Teaching Observation Protocol (RTOP), were designed prior to the development of the Next Generation Science Standards and the integration of engineering into science standards. These instruments were also designed for use in reform-based science classrooms, not engineering or integrated STEM learning environments. While engineering-focused observation protocols do exist for K-12 classrooms, they do not evaluate beyond an engineering focus, making them limited tools to evaluate integrated STEM instruction. In order to facilitate the implementation of integrated STEM in K-12 classrooms and the development of the nascent integrated STEM education literature, our research team is developing a new integrated STEM observation protocol for use in K-12 science and engineering classrooms. This valid and reliable instrument will be designed for use in a variety of educational contexts and by different education stakeholders to increase the quality of K-12 STEM education. At the end of this project, the STEM-OP will be made available through an online platform that will include an embedded training program to facilitate its broad use. In the first year of this four-year project, we are working on the initial development of the STEM-OP through video analysis and exploratory factor analysis. We are utilizing existing classroom video from a previous project with approximately 2,000 unique classroom videos representing a variety of grade levels (4-9), science content (life, earth, and physical science), engineering design challenges, and school demographics (urban, suburban). The development of the STEM-OP is guided by published frameworks that focus on providing quality K-12 integrated STEM and engineering education, such as the Framework for Quality K-12 Engineering Education. Our anticipated results at the time the ASEE meeting will include a review of our item development process and finalized items included on the draft STEM-OP. Additionally, we anticipate being able to share findings from the exploratory factor analysis (EFA) on our video-coded data, which will identify distinct instructional dimensions responsible for integrated STEM instruction. We value the opportunity to gather feedback from the engineering education community as the integration of engineering design and practices is integral to quality integrated STEM instruction. 

   more » « less
5. Integrating computational thinking into middle school science through co-designed storylines

   Biddy, Q. ; Gendreau Chakarov, A. ; Jacobs, J. ; Recker, M. ; Sumner, T. ; Penuel, W. ( January 2020 , Association for Science Teacher Education)

   We describe a professional development model that supports teachers to integrate computational thinking (CT) and computer science principles into middle school science and STEM classes. The model includes the collaborative design (co-design) (Voogt et al., 2015) of storylines or curricular units aligned with the Next Generation Science Standards (NGSS Lead States, 2013) that utilize programmable sensors such as those contained on the micro:bit. Teachers spend several workshops co-designing CT-integrated storylines and preparing to implement them with their own students. As part of this process, teachers develop or modify curricular materials to ensure a focus on coherent, student driven instruction through the investigation of scientific phenomena that are relevant to the students and utilize sensor technology. Teachers implement the storylines and meet to collaboratively reflect on their instructional practices as well as their students’ learning. Throughout this cyclical, multi-year process, teachers develop expertise in CT-integrated science instruction as they plan for and use instructional practices that align with three dimension science teaching and foreground computational thinking. Throughout the professional learning process, teachers alternate between wearing their “student hats” and their “teacher hats”, in order to maintain both a student and teacher perspective as they co-design and reflect on their implementation of CT-integrated units. This paper illustrates two teachers’ experiences of the professional development process over a two-year period, including their learning, planning, implementation, and reflection on two co-designed units. 

   more » « less