As teachers strive to increase the effectiveness of their curriculum, they may seek out professional development opportunities that focus on curriculum instruction. Guskey (1986) defines professional development as an organized effort that is geared towards changing the instructional practices of teachers with hopes that these changes will positively impact student learning (Dana et al. 1997;Kardash 2000;Loucks-Horsley et al. 2003;Seymour et al. 2003;Smith & Southerland 2007). However, the efficacy of professional development is ultimately based on the implementation of the strategies learned, which relies upon the motivation of the teachers to do so.
Unfortunately, it is quite common that teachers have low motivation to incorporate the strategies they learned resulting in low implementations (Guskey 2002). This may be attributed to the fact that the translation of the strategies learned in professional development activities is often more time-consuming than anticipated (Fullan 1993;Guskey 2002) or they attend the professional development during the school year when they lack the time to implement it. One way to combat this problem is by giving the teachers more responsibility and ownership of strategies, increasing the likelihood of their implementation in the classrooms and the resulting impact on student learning (Grove et al. 2009).
The National Science Foundation (NSF) funded Research Experiences for Teachers (RET) programs provide opportunities for professional development for teachers across the country. These programs are intended to impact the understanding of the participating teachers and thus, their ability to teach science, technology, engineering and mathematics (STEM) subjects through meaningful, real-world experiences (Faber et al. 2014;Kardash 2000;National Research Council 1996;Seymour et al. 2002;Zubrowski 2007). Most of these programs will place teachers in a laboratory at a university or with an industry partner for six to eight weeks. Teachers are expected to be immersed in scientific research so that the procedural knowledge they gain can be transferred to the students in their classrooms (Deci & Ryan 2000;Driscoll 2005;Grove et al. 2009;Hashweh, 2003;Loucks-Horsley et al. 2003;Supovitz & Turner 2002;Wenglinsky 2000;Woolfolk 2007).
The primary goal of this paper is to assess the effectiveness of the professional development provided to secondary educators that participated in the first year of the RET site established at North Dakota State University (NDSU) in their preparedness to teach STEM topics in their classrooms. This paper will begin with a short summary of the RET programming provided to nine participants that comprised the first cohort at the NDSU site in Summer 2021.
Details from the pre-and post-experience surveys along with the self-interview questions will be provided and the results of these assessment tools will be used to describe the effectiveness of the professional development activities.
The RET site at NDSU was established in May 2020 with funding from NSF (NSF Awards #1953102 and #2224135). The goal of this site was to increase the knowledge of secondary (6th to 12th grade) educators regarding the use of civil engineering to mitigate natural disasters and in turn, their ability to prepare their students to become leaders in STEM disciplines. Due to the COVID-19 pandemic and the proximity of the award start date to the summer session, the first cohort of RET participants at the NDSU site was during Summer 2021.
Teachers were engaged in hands-on authentic research experiences revolving around the theme of mitigating natural disasters in the labs of the NDSU Civil and Environmental Engineering (CEE; now Civil, Construction and Environmental Engineering or CCEE) faculty during the summer. Their activities allowed them to bridge research experiences to improve content knowledge, with the intent to improve secondary STEM education in their classrooms (Farrell 1992;Dubner et al. 2011;Silverstein et al. 2009). Over the course of the six-week program, the teachers also developed curriculum modules and interacted with the local civil engineering industry while making meaningful connections to continue to enhance STEM education in their classrooms. During the summer program, the teachers were also engaged in cohort building activities to develop an on-going regional support network.
Figure 1 provides an overview of the six-week summer experience provided at the NDSU RET site. As seen from Figure 1, the program kicks off with an orientation session and culminates in a summer capstone symposium.
During the capstone symposium, the teachers delivered two presentations -a poster presentation based on the research they conducted and an oral presentation on the curriculum module they developed based on that research.
Each week of the RET program was focused on assisting the teachers towards preparing the curriculum modules and their poster presentation, shown in the blue and orange boxes, respectively, in Figure 1. Methodology Participants The NDSU RET site accepted both in-service and pre-service teachers. In-service teachers were recruited from Fargo Public Schools, West Fargo Public Schools and South East Education Cooperation, while pre-service teachers were recruited from the NDSU School of Education. Announcements for the RET program were sent through publicly available emails found through the North Dakota Department of Public Instruction, posted on the NDSU RET webpage and distributed via contacts in school districts and the School of Education. Applications were released in mid-January with an application deadline at the end of March. An online information session was held in February to share additional details and answer any questions about the program including the application process.
Applications were reviewed by the authors to select ten teachers to participate in the NDSU RET site. Selections were made based on how well the courses each teacher taught aligned with the proposed research projects.
Underrepresented and underserved groups in STEM fields including women, Hispanic, African American or Native American individuals, veterans, individuals from rural communities and individuals with disabilities were given preference. Preference was also given to in-service teachers with at least two years of teaching experience in order to establish good mentorship between in-service and pre-service teachers.
The Summer 2021 RET Cohort consisted of one pre-service and eight in-service teachers. Table 1 summarizes demographic information about the participants. The selected teachers ranged in age from 21 years to 61 years with a mean of 42.1 years and had teaching experiences between 2 years and 33 years with a mean of 18.8 years.
Figure 2 summarizes the demographic breakdowns of the schools in the 2019-2020 academic year and provides a comparison with the demographics in the State of North Dakota during the 2010 census. It is evident that these schools serve large populations of underserved and underrepresented groups including females, Hispanics, African Americans, and Native American students. Data Collection and Analysis Procedures The data presented in this paper is based on the responses to two online surveys and a self-interview conducted with the secondary educators in the Summer 2021 NDSU RET cohort. The pre-experience survey was administered on June 14, 2021, during the orientation session of the summer program. The post-experience survey was available to the participants from July 29, 2021 to September 7, 2021. These surveys posed questions geared towards obtaining an understanding of student engagement with STEM and specifically, civil engineering topics, along with the knowledge and interest of the teachers in teaching these topics. Additional questions that focused on these issues in relation to civil engineering were also included in both surveys. The post-experience survey also gathered evaluation data on the impact of the NDSU RET program. Evaluation and reflections of the NDSU RET program are not presented in this paper but are available in Crary and Ajmera (2023).
Both the pre-and post-experience surveys contained quantitative items that assessed student engagement, attitudes, and knowledge. These questions were asked on a six-point Likert scale with a range of 1 (strongly disagree) to 6 (strongly agree
). A series of paired sample t-tests were used to analyze these items. If the paired sample t-test was significant at p<0.05 level, then Cohen's d was calculated to determine the effect size. In addition to the quantitative questions, several open-ended qualitative questions were also posed in the surveys. The openended questions are summarized in Table 2. All nine (100%) of the secondary educators participated in the preexperience survey, but only eight (88.9%) of them participated in the post-experience survey. The teachers were asked to record themselves using their cell phones as they responded to these five questions.
Each recording of the self-interview lasted approximately three minutes and were transcribed verbatim by an external evaluator. Next the external evaluator coded the data into the primary themes around which the responses to each question were analyzed.
The pre-and post-experience surveys had nine Likert scale questions related to student engagement. The preexperience survey also had three open-ended questions that aimed to gather data regarding student interest in STEM topics, barriers and challenges that the teachers faced when trying to engage students in STEM topics and some of the strategies and techniques that they found useful to address these barriers and challenges. This section will summarize the results obtained.
The results from the pre-experience Likert scale surveys indicate that the teachers felt that their students were more interested in learning about STEM rather than civil engineering topics (Table 3). In general, they were also more confident in their abilities to engage students in STEM topics in comparison to civil engineering topics.
Responses to the open-ended questions indicated that teachers felt that this may be the case because STEM lessons usually involve hands-on activities and these subjects connect with real-life situations better than non-STEM subjects. Several of the responses also indicated that their perceptions of student interests in STEM may also be biased by the courses that they teach. That is, several of the participants taught AP science and mathematics courses, which tend to have higher enrollments of students that are more inclined towards STEM topics than the regular sections of the same classes. 3) issues with open-endedness and time to get a response. Nearly all the teachers noted that projects or experiments and simulations with data were great strategies that they found helpful in engaging their students in STEM topics. In addition, they found value in connecting these topics with real-world problems or activities in the student's lives.
A comparison of the post-experience survey results with those from the pre-experience surveys indicated that the RET program was successful in making STEM and civil engineering topics more interesting for students in the opinion of the teachers, teaching teachers how to engage their students with these concepts, providing them with experiences in positively engaging their students, increasing their confidence with their abilities to engage students, and in providing them with training on how to best engage their students in STEM topics. Additionally, the teachers indicated that they felt that they would experience fewer challenges when trying to engage their students in STEM topics.
The data in Table 3 indicates that the differences between the pre-and post-experience surveys were significant for two questions. Specifically, the teachers felt that the RET program helped to engage students in learning about civil engineering and that they received training on how to best engage students in STEM concepts. The oral responses provided contextual background regarding these improvements because of the RET program. These responses were focused on three areas related to how the teachers would engage their students differently after having participated in the RET program. First, the teachers stated that they would make their students aware of the possible jobs in STEM. The teachers also stated that they would have their students engage in hands-on projects focusing on topics that allow students the opportunity to both ask critical questions related to different research aspects and think like an engineer.
The response from the self-interview also highlighted specific approaches that teachers would take to better teach their students about STEM disciplines and topics. Again, three themes appeared in their responses, which were to be more intentional with the class content, to incorporate units or lessons that were developed during the RET program, and to provide more real-world examples of problems that students could solve. In each of these, the teachers highlighted that they would use the experiences that they had obtained from their participation in the RET program to improve their teaching. More interestingly, several of the teachers expanded their responses to state that they would also use the lessons and units developed by their cohort members in their own classes.
Teachers answered nine Likert-scale questions related to attitudes towards STEM topics and disciplines in both the pre-and post-experience surveys. While neither survey had any open-ended questions regarding this topic, two of the five self-interview questions were focused on attitudes towards STEM topics and disciplines. The results from the Likert-scale questions in the pre-and post-experience surveys are summarized in Table 4 and are provided additional context based on the results from the self-interviews in this section. It is noted that all the results are based on teacher perceptions.
Results in Table 4 illustrate an improvement in the responses to all the questions. This indicates that as a result of their participation in the RET program, the teachers believed students were more interested in learning about and pursuing STEM disciplines. Furthermore, the teachers also had positive attitudes towards their belief in students pursuing STEM careers and the relevance of these topics to student lives. Finally, there was also a slight increase in the enjoyment of teachers when teaching STEM topics. Two questions were related to student interest in STEM careers and the positive attitudes that students have regarding STEM disciplines had significant results (Table 4). However, none of the questions regarding the teachers' attitudes had significant results. The responses that the teachers provided to the first self-interview question regarding the impact of the RET program on their interests in STEM disciplines and topics provide some context as to the results in Table 4. In particular, the responses clearly stated that participation in the RET program did not change teacher interest because the teachers were already very interested in STEM (and specifically, civil engineering) topics and disciplines.
The significant results may be attributed to an increased awareness of the teachers to STEM (and specifically, civil engineering) disciplines and topics (Table 4). In response to the second self-interview question, the teachers highlighted that the RET program made them more aware of the specific STEM and civil engineering topics as well as the jobs and specifics of what civil engineering entailed. In particular, teachers noted that the program opened their eyes to see the various aspects of civil engineering beyond traditional ideas like building bridges and roads. Thus, the authors can conclude that this increased awareness of the specifics of civil engineering allows teachers to better see how their students might fit into STEM careers or how they can impact their students' attitudes regarding STEM topics.
Both the pre-and post-experiences surveys administered to the teachers contained eight Likert-scale questions regarding student and teacher knowledge of STEM topics. In addition, the pre-experience survey featured two open-ended questions related to the teachers' knowledge of STEM topics. The first question focused on strategies and techniques that the teachers found helpful to develop their knowledge of STEM topics, while the second asked teachers what skills and knowledge that they wanted to gain to feel more prepared to teach STEM topics to their students. Finally, the impact of the RET program on developing teacher knowledge regarding STEM and civil engineering topics was ascertained through one question in the self-interviews at the end of the summer program.
The results from all the assessments are summarized in this section.
Table 5 summarizes the mean and standard deviations of each Likert-scale question on the pre-and postexperience surveys. Additionally, it also summarizes the results of a paired t-test and indicates which were statistically significant. Cohen's d is provided for t-tests with statistically significant p-values. The differences in the responses from the teachers were statistically significant in over half of the questions illustrating the substantial impact that participation in the RET program had on the teachers' and their perceptions of students' knowledge of STEM topics.
It is evident that teachers believed that their students were more knowledgeable about STEM topics than those in civil engineering (Table 5) based on the results. While both questions saw statistically significant improvement in the levels of agreement following the participation of the teachers in the RET program, students were still seen to possess greater STEM topic knowledge than civil engineering knowledge. The teachers also indicated significant differences regarding their familiarity with current civil engineering research, the professional development that they had received to teach STEM topics and the knowledge that they needed to teach STEM topics as a result of their participation in the RET program.
Based on the responses provided by the teachers to the open-ended question regarding strategies and techniques that they used to develop their knowledge of STEM topics in the pre-experience survey, it was apparent that the primary mechanism that teachers had to develop their abilities to teach STEM topics was via collaborations with other teachers both within and outside of their district. They appeared to rely on working together to develop new activities or to borrow activities that others had already implemented in their classrooms. Two teachers also noted that they relied on their own experiences from working in the field or individual continuing education that they sought out. Given this, the results regarding the professional development that the teachers have received to teach STEM topics in Table 5 are not surprising. The RET program was specifically designed to provide teachers with these professional development opportunities and it is clear that the program is filling a major gap in this area. The paired t-test results regarding teacher knowledge to teach about STEM topics indicated significance within three themes. First, the teachers wanted to gain more knowledge and experience in order to feel more prepared to teach STEM topics. Specifically, the teachers highlighted that they wanted to learn more about civil engineering principles and how the classes they taught were connected to these principles. Additionally, the teachers hoped to walk away with good ideas for real-world applications of the content that they teach, with a specific focus on the local region or the work being done at NDSU. Finally, several of the teachers also expressed a desire to find ways to make the basics that they taught more fun and interesting for their students. From the themes identified from the responses from the self-interview, it was clear that some of these desires were achieved. Teachers stated that they had increased knowledge about the research projects that were being undertaken at NDSU and around the country. They also stated that they gained specific knowledge regarding STEM topics and disciplines, with a focus on how civil engineering differs from other engineering fields. The teachers stated that their knowledge was further increased by their interactions with other teachers in the RET program as well as the presentations that the other teachers delivered at various points during the summer activities. Responses to other questions on the selfinterview indicated that the teachers did learn about various real-world applications and other ways to engage their students in STEM, and specifically, civil engineering, topics, and disciplines. Finally, the post-experience survey also included several questions in which the teachers evaluate the RET program. This evaluation asked two questions related to teachers' knowledge of STEM and civil engineering disciplines. All the teachers either agreed or strongly agreed that the RET program increased their knowledge of both STEM and civil engineering topics.
The NSF RET experience offered at NDSU in the summer of 2021 had a positive impact on improving civil engineering and STEM lessons in the secondary school setting. As this program continues for at least two more cohorts, it is important to continue to foster the interaction between the teachers as well as with the faculty. As the participants return to their classroom their students will benefit from the improved understanding of their teachers as well as the authentic curriculum units, inspiring future engineers with possible careers.