skip to main content


Title: Evaluation of Engineering Problem-framing Professional Development for K-12 Science Teachers
As the importance to integrate engineering into K12 curricula grows so does the need to develop teachers’ engineering teaching capabilities and knowledge. One method that has been used to aid this development is engineering professional development programs. This evaluation paper presents the successes and challenges of an engineering professional development program for teachers focused around the use of engineering problem-framing design activities in high school science classrooms. These activities were designed to incorporate the cross-cutting ideas published in the Next Generation Science Standards (NGSS) and draw on best practices for instructional design of problem-framing activities from research on design and model-eliciting activities (MEAs). The professional development (PD) was designed to include the following researched-based effective PD key elements: (1) is content focused, (2) incorporates active learning, (3) supports collaboration, (4) uses models of effective practice, (5) provides coaching and expert support, (6) offers feedback and reflection, and (7) is of sustained duration. The engineering PD, including in-classroom deployment of activities and data collection, was designed as an iterative process to be conducted over a three-year period. This will allow for improvement and refinement of our approach. The first iteration, reported in this paper, consisted of seven high school science teachers who have agreed to participate in the PD, implement the problem-framing activities, and collect student data over a period of one year. The PD itself consisted of the teachers comparing science and engineering, participating in problem-framing training and activities, and developing a design challenge scenario for their own courses. The participating teachers completed a survey at the end of the PD that will be used to inform enhancement of the PD and our efforts to recruit additional participants in the following year. The qualitative survey consisted of open-ended questions asking for the most valuable takeaways from the PD, their reasoning for joining the PD, reasons they would or would not recommend the PD, and, in their opinion, what would inspire their colleagues to attend the PD. The responses to the survey along with observations from the team presenting the PD were analyzed to identify lessons learned and future steps for the following iteration of the PD. From the data, three themes emerged: Development of PD, Teacher Motivation, and Teacher Experience.  more » « less
Award ID(s):
1812823
NSF-PAR ID:
10173425
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
2020 ASEE Virtual Annual Conference
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Objective Over the past decade, we developed and studied a face-to-face video-based analysis-of-practice professional development (PD) model. In a cluster randomized trial, we found that the face-to-face model enhanced elementary science teacher knowledge and practice and resulted in important improvements to student science achievement (student treatment effect, d = 0.52; Taylor et al, 2017; Roth et al, 2018). The face-to-face PD model is expensive and difficult to scale. In this paper, we present the results of a two-year design-based research study to translate the face-to-face PD into a facilitated online PD experience. The purpose is to create an effective, flexible, and cost-efficient PD model that will reach a broader audience of teachers. Perspective/Theoretical Framework The face-to-face PD model is grounded in situated cognition and cognitive apprenticeship frameworks. Teachers engage in learning science content and effective science teaching practices in the context in which they will be teaching. There are scaffolded opportunities for teachers to learn from analysis of model videos by experienced teachers, to try teaching model units, to analyze video of their own teaching efforts, and ultimately to develop their own unit, with guidance. The PD model attends to the key features of effective PD as described by Desimone (2009) and others. We adhered closely to the design principles of the face-to-face model as described by Authors, 2019. Methods We followed a design-based research approach (DBR; Cobb et al., 2003; Shavelson et al., 2003) to examine the online program components and how they promoted or interfered with the development of teachers’ knowledge and reflective practice. Of central interest was the examination of mechanisms for facilitating teacher learning (Confrey, 2006). To accomplish this goal, design researchers engaged in iterative cycles of problem analysis, design, implementation, examination, and redesign (Wang & Hannafin, 2005) in phase one of the project before studying its effect. Data Three small pilot groups of teachers engaged in both synchronous and asynchronous components of the larger online course which began implementation with a 10-week summer course that leads into study groups of participants meeting through one academic year. We iteratively designed, tested, and revised 17 modules across three pilot versions. On average, pilot groups completed one module every two weeks. Pilot 1 began the work in May 2019; Pilot 2 began in August 2019, and Pilot 3 began in October 2019. Pilot teachers responded to surveys and took part in interviews related to the PD. The PD facilitators took extensive notes after each iteration. The development team met weekly to discuss revisions. We revised all modules between each pilot group and used what we learned to inform our development of later modules within each pilot. For example, we applied what we learned from testing Module 3 with Pilot 1 to the development of Module 3 for Pilots 2, and also applied what we learned from Module 3 with Pilot 1 to the development of Module 7 for Pilot 1. Results We found that community building required the same incremental trust-building activities that occur in face-to-face PD. Teachers began with low-risk activities and gradually engaged in activities that required greater vulnerability (sharing a video of themselves teaching a model unit for analysis and critique by the group). We also identified how to contextualize technical tools with instructional prompts to allow teachers to productively interact with one another about science ideas asynchronously. As part of that effort, we crafted crux questions to surface teachers’ confusions or challenges related to content or pedagogy. We called them crux questions because they revealed teachers’ uncertainty and deepened learning during the discussion. Facilitators leveraged asynchronous responses to crux questions in the synchronous sessions to push teacher thinking further than would have otherwise been possible in a 2-hour synchronous video-conference. Significance Supporting teachers with effective, flexible, and cost-efficient PD is difficult under the best of circumstances. In the era of covid-19, online PD has taken on new urgency. NARST members will gain insight into the translation of an effective face-to-face PD model to an online environment. 
    more » « less
  2. This fundamental research in pre-college education engineering study investigates the ways in which elementary teachers learn about engineering by engaging in the epistemic practices of engineers. Teaching engineering explicitly in elementary settings is a paradigm shift, as most K-6 teachers are not taught about engineering in their preparation programs and did not do classroom engineering as students. However, current STEM education reforms require these teachers to teach engineering in science settings and it will require concerted efforts between professional development providers and educational researchers to better help these teachers learn about and teach engineering to their students. Our study context consisted of 18 2nd and 4th grade teachers participating in one of two two-day workshops. The first day focused on what engineering is, what the epistemic practices of engineering are, and how to manage classroom engineering projects. The second day focused on how to teach a specific engineering unit for their grade level. Taking a sociomaterial view of learning, we asked the following research questions: 1. How do the engineering notebooks scaffold the teachers activities and discourse? 2. How and to what extent does the notebook support their engagement in engineering practices? Our analysis triangulated between three data sources during a two-hour time period where teachers designed, tested, and improved enclosures intended to minimize cost and mass loss of an ice cube in a heat chamber (“Perspiring Penguins” (Schnittka, 2010)). We focused on teacher talk/action collected from video/audio recordings trained on four small groups (10 total teachers). We also collected engineering notebooks they used during this activity. After initial analyses, we followed up with select teachers with targeted interview questions to focus on clarification of questions that arose. Our findings suggest that the teachers use the notebooks in ways that are significantly different from the ways engineers do; however, they are a useful pedagogical tool that supported them in attending to and discussing activities that were necessary to engage in engineering practices and design/re-design their technology. Additionally, our paper will describe specific examples where teachers had rich discussions that were not represented in the notebooks but there were references made in the notebooks that were not explicitly discussed. Implications for the importance of well-designed notebooks and the benefits of ethnographic methods for researching teacher learning will be discussed. 
    more » « less
  3. Objective Over the past decade, we developed and studied a face-to-face video-based analysis-of-practice PD model. In a cluster randomized trial, we found that the face-to-face model enhanced elementary science teacher knowledge and practice, and resulted in important improvements to student science achievement (student treatment effect, d = 0.52; Taylor et al., 2017: Roth et al., 2018). The face-to-face PD model is expensive and difficult to scale. In this poster, we present the results of a two-year design-based research study to translate the face-to-face PD into a facilitated online PD experience. The purpose is to create an effective, flexible, and cost-efficient PD model that will reach a broader audience of teachers. Perspective/Theoretical Framework The face-to-face PD model is grounded in situated cognition and cognitive apprenticeship frameworks. Teachers engage in learning science content and practices in the context in which they will be teaching. In addition, there are scaffolded opportunities for teachers to learn from model videos by experienced teachers, try model units, and ultimately develop their own unit, with guidance. The PD model also attends to the key features of effective PD as described by Desimone (2009) and others. We adhered closely to the design principles of the face-to-face model as described by Roth et al., 2018. Methods We followed a design-based research approach (DBR: Cobb et al., 2003: Shavelson et al., 2003) to examine the online program components and how they promoted or interfered with the development of teachers’ knowledge and reflective practice. Of central interest was the examination of mechanisms for facilitating teacher learning (Confrey, 2006). To accomplish this goal, design researchers engaged in iterative cycles of problem analysis, design, implementation, examination, and redesign (Wang & Hannafin, 2005). Data We iteratively designed, tested, and revised 17 modules across three pilot versions. Three small groups of teachers engaged in both synchronous and asynchronous components of the larger online course. They responded to surveys and took part in interviews related to the PD. The PD facilitators took extensive notes after each iteration. The development team met weekly to discuss revisions. Results We found that community building required the same incremental trust-building activities that occur in face-to-face PD. Teachers began with low-risk activities and gradually engaged in activities that required greater vulnerability (sharing a video of themselves teaching a model unit for analysis and critique by the group). We also identified how to contextualize technical tools with instructional prompts to allow teachers to productively interact with one another about science ideas asynchronously. As part of that effort, we crafted crux questions to surface teachers’ confusions or challenges related to content or pedagogy. Facilitators leveraged asynchronous responses to crux questions in the synchronous sessions to push teacher thinking further than would have otherwise been possible in a 2-hour synchronous video-conference. Significance Supporting teachers with effective, flexible, and cost-efficient PD is difficult under the best of circumstances. In the era of COVID-19, online PD has taken on new urgency. AERA members will gain insight into the construction of an online PD for elementary science teachers/ Full digital poster available at: https://aera21-aera.ipostersessions.com/default.aspx?s=64-5F-86-2E-15-F8-C3-C0-45-C6-A0-B7-1D-90-BE-46 
    more » « less
  4. Teacher self-efficacy (SE) has been observed to be an 'important construct for Computer Science (CS) teachers' professional development because it can predict both teaching behaviors as well as student outcomes" [1]. The purpose of the present study was to investigate teacher CS SE during a two-year federally funded professional development (PD) and curriculum development project for middle school teachers incorporating game-design and the Unity development platform. The research question investigated is: How does teacher self-efficacy for teaching computer science via game design with the Unity game development platform change during a year-long PD program? Investigations of teacher SE for teaching CS have resulted in some surprising results. For example, it has been reported that - There were no differences in self-efficacy based on teachers' overall level of experience, despite previous findings that teacher self-efficacy is related to amount of experience" and "no differences in self-efficacy related to the teachers' own level of experience with CS" [2], thus further study of CS teacher SE is warranted. Participants in this study were six middle school teachers from four middle schools in the southeastern United States. They participated in a year-long PD program learning the Unity game development platform, elements of game design, and foundations of learner motivation. Guided reflective journaling was used to track the teachers' SE during the first year of the project. Teachers completed journal prompts at four intervals. Prompts consisted of questions like "How do you currently feel about your ability to facilitate student learning with Unity?" and "Are you confident that you can implement the materials the way the project team has planned for them to be implemented?" Prior to beginning the project participants expressed confidence in being able to facilitate student learning after participating in the planned professional development, but there was some uneasiness about learning and using Unity. From a SE perspective their responses make sense, as all of the participants are experienced teachers and should have confidence in their general ability to teach. However, since Unity is a new programming environment for all of the teachers, they did not have the prior experience necessary to have a high degree of confidence that they could successfully use it with their students. 
    more » « less
  5. Biologically inspired design has become increasingly common in graduate and undergraduate engineering programs, consistent with an expanding emphasis by professional engineering societies on cross-disciplinary critical thinking skills and adaptive and sustainable design. However, bio-inspired engineering is less common in K-12 education. In 2019, the NSF funded a K-12 project entitled Biologically Inspired Design for Engineering Education (BIRDEE), to create socially relevant, accessible, and highly contextualized high school engineering curricula focusing on bio-inspired design. Studies have shown that women and underrepresented minorities are drawn to curricula, courses, and instructional strategies that are integrated, emphasize systems thinking, and facilitate connection building across courses or disciplines. The BIRDEE project also seeks to interest high school girls in engineering by providing curricula that incorporate humanistic, bio-inspired engineering with a focus on sustainable and authentic design contexts. BIRDEE curricula integrate bio-inspired design into the engineering design process by leveraging design tools that facilitate the application of biological concepts to design challenges. This provides a conceptual framework enabling students to systematically define a design problem, resulting in better, more well-rounded problem specifications. The professional development (PD) for the participating teachers include six-week-long summer internships in university research laboratories focused on biology and bio-inspired design. The goal of these internships is to improve engineering teachers’ knowledge of bio-inspired design by partnering with cutting-edge engineers and scientists to study animal features and behaviors and their applications to engineering design. However, due to COVID-19 and research lab closures in the summer of 2020, the research team had to transfer the summer PD experience to an online setting. An asynchronous, quasi-facilitated online course was developed and delivered to teachers over six weeks. In this paper, we will discuss online pedagogical approaches to experiential learning, teaching bio-inspired design concepts, and the integration of these approaches in the engineering design process. Central to the online PD design and function of each course was the use of inquiry, experiential and highly-collaborative learning strategies. Preliminary results show that teachers appreciated the aspects of the summer PD that included exploration, such as during the “Found Object” activity, and the process of building a prototype. These activities represented experiential learning opportunities where teachers were able to learn by doing. It was noted throughout the focus group discussions that such opportunities were appreciated by participating teachers. Teachers indicated that the experiential learning components of the PD allowed them to do something outside of their comfort zone, inspired them to do research that they would not have done outside of this experience, and allowed them to “be in the student's seat and get hands-on application”. By participating in these experiential learning opportunities, teachers were also able to better understand how the BIRDEE curriculum may impact students’ learning in their classrooms 
    more » « less