Despite limited success in broadening participation in engineering with rural and Appalachian youth, there remain challenges such as misunderstandings around engineering careers, misalignments with youth's sociocultural background, and other environmental barriers. In addition, middle school science teachers may be unfamiliar with engineering, may not know how to integrate engineering concepts into science lessons, or may not have the time or resources to develop such curriculum. With good intention, the resulting attempts to broaden participation may be single activities such as a professional development workshop for teachers or a career day for students. Though these may introduce a teacher or student to engineering, they are less likely to provide sustained improvements in terms of broadening participation or decreasing misalignments of engineering. In addition, single interventions are unlikely to cause significant improvement in teacher confidence to teach engineering. In an effort to improve teacher confidence of engineering curriculum and to reduce teacher and student mis-conceptions of engineering, this NSF funded ITEST project used a collaborative model to provide industry and University support to middle school science teachers to 1) develop approximately monthly science activities (curriculum) with a contextually relevant engineering component, 2) provide local engineering support in the classroom, and 3) provide financial resources to support the activities. Now in our third and final year, this paper focuses on the project's accomplishments to date in each of four key areas: 1) collaboration development, 2) student perceptions of engineers and engineering, 3) teacher confidence and self-efficacy regarding teaching engineering, and 4) curriculum development.
To address these challenges, we have undertaken our NSF ITEST project titled, The Virginia Tech Partnering with Educators and Engineers in Rural Schools (VT PEERS). Through this project, we seek to improve youth awareness of and preparation for engineering related careers and educational pathways and improved middle school science teacher confidence to integrate engineering into their classrooms. Utilizing regular engagement in engineering-aligned classroom activities and culturally relevant programming, we sought to spark an interest with students. In addition, our project involves a partnership with teachers, school districts, and local industry to provide a holistic and, hopefully, sustainable influence. By engaging over time, as opposed to single activities, we aspire to promote sustainability of relevant engineering activities in middle school science classes beyond this NSF project via increased teacher confidence with engineering related activities and continued relationships with local industry.
The overarching goals of our research project align with the aims of the NSF ITEST program. To achieve these goals, much of our work was operationalized and focused on individual school engagements and activities. Now in our third and final full year, a key focus area is ensuring we continue to integrate our research findings into practice, in particular to support classroom activities (Goal 2) and disseminate our findings. During the first three years we have focused on understanding: 1) collaboration development, 2) student perceptions of engineers and engineering, 3) teacher confidence and self-efficacy regarding teaching engineering, and 4) curriculum development. This paper provides a summary of each of these four areas.
For the purposes of this paper, we refer to curriculum development as engineering activities lasting one class period and related to the Commonwealth of Virginia's science standards of learning (SOLs). During the course of an academic year, teachers engaged in approximately three (for semester courses) to six (for yearlong courses) engineering activities. Note that our partnering counties contained a combination of semester and year-long science classes with corresponding class periods ranging from 75-minutes to 55-minutes.
During the 2017-2021 school years, the project engaged seven schools across three rural counties in Virginia. Each academic year (through the 2019-2020) a grade level was added; that is, the first-year teachers and students remained for all three years. Year one included eight 6th grade science teachers, year two added eight 7th grade science teachers, and year three added three 8 th grade science teachers and a career and technology teacher. The number of students increased from over 500 students in year one to over 2500 in year three. Our three industry partners (companies) have remained active throughout the project. During the extension year of 2020-2021, no students or classes were added; in fact, due to the COVID-19 pandemic, we focused our engagement to supporting teachers by maintaining contact with them and aiding them as requested.
Research and programmatic frameworks.
The VT PEERS programmatic and research efforts were guided by several distinct frameworks around action research, the study of career choice, and organizational behavior. The multiple frameworks and lens were necessary to guide the overall program (including our goal of program improvement year over year), provide structure for our research methods, and to assist with our classroom activity development. Using an approach grounded in design-based implementation research (DBIR) methodologies [1,2], the VT PEERS project engaged partners through a cycle of research and practice around student, teacher, and collaborative outcomes. As previously reported [3], the overall project is also guided by the conceptual framework of the Promoting and Supporting Engineering Career Choices (PSECC) model shown in the figure below [4]. PSECC provided a framework to assist with data collection and analysis as well as a reminder for deploying culturally relevant classroom activities. Together, DBIR and PSECC were used to guide the development of relevant, forward-looking classroom activities, aide our data analysis, and improve our collaborative interactions and classroom activities year over year. In particular, DBIR guided the team as we sought to integrate our research findings into the project prior to the start of each academic year. The efforts resulted in improvements related to
• teacher efficacy and confidence relative to engineering and teaching engineering • student knowledge of and interest in engineering and engineering careers.
PSECC Model [4] While the structure of DBIR provides an approach for the overarching project, and the PSECC model provides a lens with which to design curriculum and study teacher and student outcomes, throughout the project we also utilized the ITEST STEM Workforce Education Helix model to support a pragmatic approach of our research informing our practice to enable an "iterative relationship between STEM content development and STEM career development activities… within the cultural context of schools, with teachers supported by professional development, and through programs supported by effective partnerships." [8, p. 849] For example, over the course of the project, scaffolding from the University leading activities to teachers leading activities occurred.
Detailed information for the overall VT PEERS program data collection has been published previously [3,5,6]. For both youth and adults, participation in the research was decoupled from participation in the programmatic activities, though the majority of participants in the program chose to participate in the research. Data collection was longitudinal, included quantitative and qualitative instruments, and the participants included teachers, students, industry partners, and university partners. In addition, the surveys and interviews were pre and post activities for each year. During the first year seven schools in our three counties were involved with eight teachers, approximately ten industry partners, and several graduate students (supporting the in-class activities). By year three we remained in all of our schools, have 20 teachers involved, over 1500 6 th -8 th grade students, 27 professionals from industry, and 26 graduate students.
Although we are still heavily engaged in classroom activities and research, herein, we summarize work to date across our four areas of focus: 1) collaboration development and community engagement, 2) student perceptions of engineers and engineering, 3) teacher confidence and selfefficacy regarding teaching engineering, and 4) curriculum development.
Work related to community engagement was previously reported [3,5]. Through such work, we were able to characterize and describe the collaborations. From a pragmatic perspective, it is noteworthy that our three industry partners remained with the program all three years. Each year the size of our program increased in terms of teachers and students involved, as shown in Table 1. Though we scaffolded our involvement with the activities (as discussed below), our partners supported the project with an increasing number of employees who had helped. Our community partners each had core participants, whom the students began to know. In at least one case, as noted by the research observer, 7 th grade students in year two (i.e., 6 th grade students in year one) happily greeted, by name, a core industry member in South County when she was back for their first activity of the year. We did find that the growth can present challenges for industry partners.
Specifically, though all industry partners remained actively engaged, one of the partners expressed concern at the growth of the program and their ability to support it to the level they would like.
The Draw an Engineer Test (DAET) instrument [7] was used to help assess students' changes regarding knowledge of who engineers are and what they do. Based on 232 matched pre-and post-intervention responses to the question asking if students know any engineers, we found that 28% indicated that they do know an engineer in both surveys, 43% indicated that they do not know an engineer in both surveys, 18% moved from not knowing an engineer to knowing an engineer, and 11% moved from knowing an engineer to not knowing an engineer [6,7]. Based on our analysis, we believe these numbers offer preliminary evidence that we are helping students develop concrete ideas of who engineers are and what they do.
Moreover, drawings and descriptions of engineers seemingly shifted towards representing content from the PEER. Although the drawings did generally reflect the kinds of actions and artifacts found in other studies (e.g. [9]), in our study more students represented cars, buildings, and the ideas of fixing and repair post compared to pre intervention which is different than prior works (e.g. [10]). This is consistent with our modules such as one building mountain roads where marbles represented cars traveling on said roads and a biome module that talked about engineering with regard to impacts of buildings on water run-off [6].
Teacher Self-efficacy, Confidence, and Scaffolding.
Data collection for teacher self-efficacy for leading engineering related activities continues. However, our initial findings show increases in teacher confidence in teaching engineering, but significant challenges still remain. Teachers primarily identified their role in the collaboration as supportive to the university. In year two, we increased scaffolding to encourage 6 th grade teacher independence while providing more substantial support to 7 th grade teachers during classroom activities. In year three, we have "flipped" the roles such that the teachers are responsible for leading the activities. However, teachers are welcome to invite the industry and university partners to attend and help on the days the engineering activities are being given. Based on observations made during year three, teachers are providing the bulk of the engineering activity content, though some of the engineering connections within the lessons were noted to have decreased.
Progress has also been made towards goal 2 in terms of integrating Engineering Skills and Knowledge of Diverse Engineering-Related Careers and Educational Pathways. As previously reported [3,5], our collaborative process helped develop and foster relationship building and trust. In conjunction with this relationship building, our project intentionally scaffolded from curriculum delivery primarily from the University team (year 1) to primarily the teachers (year 3).
During year 1 the majority of teachers self-reported a low level of confidence to teach engineering and a low knowledge of what engineering is and who engineers are (Authors, in work). However, by the post year one interviews, teachers reported a higher level of confidence, a realization that they knew more about engineering than they realized, and demonstrated an increased ability to provide the activities in a more independent fashion. In our third year we had a combination of teachers providing an activity without support from the university or industry partners and modifying activities to improve the logistic requirements (e.g., time between classes). The primary reason for teachers requesting support in the third year was due to the advantage of added resources in the classroom for the hands-on, open-ended activities.
As a project team, we endeavored to adhere to three priorities which were part of our initial ITEST proposal and a result of our initial, pre-intervention, collaboration meetings: 1) alignment with Virginia standards of learning, 2) introducing engineering, especially in culturally relevant ways, and 3) potential for sustainability. Over the course of the last three years, several lessons were learned directly related to curriculum. Lessons learned primarily evolved around classroom logistics and integration with already full science criteria.
In addition, we quickly realized the teaching constraints of time, cost, and space. Though the constraints varied by county, and even school in some instances, we endeavored to accommodate the lowest common denominator. Regarding time, some class periods were 50-minutes with only 5 minutes in between for set-up and clean-up. The school systems we worked with are in low socioeconomic status regions and the schools' budget for supplies are very limited, thus we worked with low-cost solutions and functionality over aesthetics -this had the added benefit of allowing students' imaginations to be used in some cases. Several of the science classrooms were very crowded and had desks, in lieu of tables, and some rooms did not have a source of water. Thus, we needed to work with our partners to provide a curriculum that could be led by the teachers, conducted during a single class period, and did not take up much space or cost.
It is important to note that the range of covered science standards was broad, including topics such as genetics, water quality, energy, space, and ecosystems. Despite this breadth, the guiding curricular priorities are consistent across the program. A curriculum example which illustrates the above three items was called Mountain Roads. With Mountain Roads, students use the engineering design process (engineering) to construct a road or path around a mountain (locallyrelevant) within given constraints. The activity provides an opportunity to design and revise a solution (open-ended) while getting first-hand experience with potential and kinetic energy (state science standards). Materials include buckets, trash bags, masking tape, marbles, and foam pipe insulation (low-cost, accessible materials). Industry partners discussed how the constraints, design processes, failure, and teamwork related to the project were relevant to their own work (locally relevant).
As part of our sustainability and broadening participation, the team researched open sources to house our curriculum plans. The team developed and included information beyond the curriculum sheets to aid teachers in using the material, and thinking about how they can tailor it to support their context and be culturally relevant. Based on our research, the curriculum is made available and easily accessible in a virtual format through the use of #GoOpenVA (https://goopenva.org/), a website sponsored through the VA Department of Education that offers openly-licensed resources for educators. This will ensure that materials are always available and can be easily located and downloaded by educators.
Over the last three years we have endeavored to meet our top-level goals. During this time, we have attempted to operationalize our goals. During what would have been our final year, COVID-19 restrictions minimized our ability to work with teachers, however we have endeavored to maintain a relationship with them and to support their engineering activities as possible. We plan to continue working with our industry and school partners in each of the four discussion areas listed above. As part of our work we are assessing what "next steps" look like post this NSF funded ITEST project.