Integrating engineering into the K‐12 science curriculum continues to be a focus in national reform efforts in science education. Although there is an increasing interest in research in and practice of integrating engineering in K‐12 science education, to date only a few studies have focused on the development of an assessment tool to measure students’ understanding of engineering design. Most of the existing measures focus only on knowledge and understanding of engineering design concepts using multiple‐choice items with the exception of the mixed‐format Engineering Concept Assessment (ECA). Also, advanced measurement models are lacking application in the testing of such mixed‐format assessments in science education. This study applied many‐faceted Rasch measurement to the modified ECA for eighth‐grade (ECA/M8) and a newly constructed rubric applied by five judges across 497 eighth‐grade students’ responses after experiencing an integrated learning unit on the engineering design process. The results supported the fit of the items and rubric rating scales to the Rasch specifications. Recommendations are made for item wording, and further reliability and validity testing of the ECA/M8, and use of the ECA/M8 in science education and research.
Exploring Student Understanding of Force and Motion Using a Simulation-Based Performance Assessment
Performance assessment (PA) has been increasingly advocated as a method for measuring students’ conceptual understanding of scientific phenomena. In this study, we describe preliminary findings of a simulation- based PA utilized to measure 8th grade students’ understanding of physical science concepts taught via an experimental problem-based curriculum, SLIDER (Science Learning Integrating Design Engineering and Robotics). In SLIDER, students use LEGO robotics to complete a series of investigations and engineering design challenges designed to deepen their understanding of key force and motion concepts (net force, acceleration, friction, balanced forces, and inertia). The simulation-based performance assessment consisted of 4 tasks in which students engaged with video simulations illustrating physical science concepts aligned to the SLIDER curriculum. The performance assessment was administered to a stratified sample of 8th grade students (N=24) in one school prior to and following implementation of the SLIDER curriculum. In addition to providing an illustration of the use of simulation- based performance assessment in the context of design-based implementation research (DBIR), the results of the study indicate preliminary evidence of student learning over the course of curriculum implementation.
more »
« less
- Award ID(s):
- 0918618
- NSF-PAR ID:
- 10376265
- Date Published:
- Journal Name:
- Journal of Research in STEM Education
- Volume:
- 2
- Issue:
- 1
- ISSN:
- 2149-8504
- Page Range / eLocation ID:
- 39 to 58
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract -
null (Ed.)This multiple case study focused on the implementation of a computer-aided design (CAD) simulation to help students engage in engineering design to learn science concepts. Our findings describe three case studies that adopted the same learning design and adapted it to three different populations, settings, and classroom contexts: at the middle-school, high-school, and pre-service teaching levels. Although the classroom orchestration of the particular learning design was customised for specific audiences and contexts, findings from this study suggest that the core components of the learning design, such as content, assessment, and pedagogy, and their alignment among them, resulted in students’ learning. Specifically, results from a pre-post science assessment suggest that the three student groups arrived at similar understanding post-intervention levels, along with a significant aggregate growth in their scientific understanding. Regarding design performance, students in different groups demonstrated different levels of success in meeting design constraints. The findings also suggest that students’ success rate in meeting the design constraints directly influenced their final design performance, where middle-school students had better performance than students in the other groups. That is, across the board, students increased their conceptual understanding of heat transfer, Earth, and solar science and were able to produce feasible designs. Implications of the study include how learning experiences with engineering and science simulations should be designed so that teachers can adopt and adapt materials for their specific audiences, contexts, and settings.more » « less
-
Background/Context: Bi/multilingual students’ STEM learning is better supported when educators leverage their language and cultural practices as resources, but STEM subject divisions have been historically constructed based on oppressive, dominant values and exclude the ways of knowing of nondominant groups. Truly promoting equity requires expanding and transforming STEM disciplines. Purpose/Objective/Research Question/Focus of Study: This article contributes to efforts to illuminate emergent bi/multilingual students’ ways of knowing, languaging, and doing in STEM. We follow the development of syncretic literacies in relation to translanguaging practices, asking, How do knowledges and practices from different communities get combined and reorganized by students and teachers in service of new modeling practices? Setting and Participants: We focus on a seventh-grade science classroom, deliberately designed to support syncretic literacies and translanguaging practices, where computer science concepts were infused into the curriculum through modeling activities. The majority of the students in the bilingual program had arrived in the United States at most three years before enrolling, from the Caribbean and Central and South America. Research Design: We analyze one lesson that was part of a larger research–practice partnership focused on teaching computer science through leveraging translanguaging practices and syncretic literacies. The lesson was a modeling and computing activity codesigned by the teacher and two researchers about post–Hurricane María outmigration from Puerto Rico. Analysis used microethnographic methods to trace how students assembled translanguaging, social, and schooled practices to make sense of and construct models. Findings/Results: Findings show how students assembled representational forms from a variety of practices as part of accomplishing and negotiating both designed and emergent goals. These included sensemaking, constructing, explaining, justifying, and interpreting both the physical and computational models of migration. Conclusions/Recommendations: Implications support the development of theory and pedagogy that intentionally make space for students to engage in meaning-making through translanguaging and syncretic practices in order to provide new possibilities for lifting up STEM learning that may include, but is not constrained by, disciplinary learning. Additional implications for teacher education and student assessment practices call for reconceptualizing schooling beyond day-to-day curriculum as part of making an ontological shift away from prioritizing math, science, and CS disciplinary and language objectives as defined by and for schooling, and toward celebrating, supporting, and centering students’ diverse, syncretic knowledges and knowledge use.more » « less
-
This project was designed to address three major challenges faced by undergraduate engineering students (UES) and pre-service teachers (PSTs): 1) retention for UESs after the first year, and continued engagement when they reach more difficult concepts, 2) to prepare PSTs to teach engineering, which is a requirement in the Next Generation Science Standards as well as many state level standards of learning, and 3) to prepare both groups of students to communicate and collaborate in a multi-disciplinary context, which is a necessary skill in their future places of work. This project was implemented in three pairs of classes: 1) an introductory mechanical engineering class, fulfilling a general education requirement for information literacy and a foundations class in education, 2) fluid mechanics in mechanical engineering technology and a science methods class in education, and 3) mechanical engineering courses requiring programming (e.g., computational methods and robotics) with an educational technology class. All collaborations taught elementary level students (4th or 5th grade). For collaborations 1 and 2, the elementary students came to campus for a field trip where they toured engineering labs and participated in a one hour lesson taught by both the UESs and PSTs. In collaboration 3, the UESs and PSTs worked with the upper-elementary students in their school during an after school club. In collaborations 1 and 2, students were assigned to teams and worked remotely on some parts of the project. A collaboration tool, built in Google Sites and Google Drive, was used to facilitate the project completion. The collaboration tool includes a team repository for all the project documents and templates. Students in collaboration 3 worked together directly during class time on smaller assignments. In all three collaborations lesson plans were implemented using the BSCS 5E instructional model, which was aligned to the engineering design process. Instruments were developed to assess knowledge in collaborations 1 (engineering design process) and 3 (computational thinking), while in collaboration 2, knowledge was assessed with questions from the fundamentals of engineering exam and a science content assessment. Comprehensive Assessment of Team Member Effectiveness (CATME) was also used in all 3 collaborations to assess teamwork across the collaborations. Finally, each student wrote a reflection on their experiences, which was used to qualitatively assess the project impact. The results from the first full semester of implementation have led us to improvements in the implementation and instrument refinement for year 2.more » « less
-
Despite limited success in broadening participation in engineering with rural and Appalachian youth, there remain challenges such as misunderstandings around engineering careers, misalignments with youth’s sociocultural background, and other environmental barriers. In addition, middle school science teachers may be unfamiliar with engineering or how to integrate engineering concepts into science lessons. Furthermore, teachers interested in incorporating engineering into their curriculum may not have the time or resources to do so. The result may be single interventions such as a professional development workshop for teachers or a career day for students. However, those are unlikely to cause major change or sustained interest development. To address these challenges, we have undertaken our NSF ITEST project titled, Virginia Tech Partnering with Educators and Engineers in Rural Schools (VT PEERS). Through this project, we sought to improve youth awareness of and preparation for engineering related careers and educational pathways. Utilizing regular engagement in engineering-aligned classroom activities and culturally relevant programming, we sought to spark an interest with some students. In addition, our project involves a partnership with teachers, school districts, and local industry to provide a holistic and, hopefully, sustainable influence. By engaging over time we aspired to promote sustainability beyond this NSF project via increased teacher confidence with engineering related activities, continued integration within their science curriculum, and continued relationships with local industry. From the 2017-2020 school years the project has been in seven schools across three rural counties. Each year a grade level was added; that is, the teachers and students from the first year remained for all three years. Year 1 included eight 6th grade science teachers, year 2 added eight 7th grade science teachers, and year 3 added three 8th grade science teachers and a career and technology teacher. The number of students increased from over 500 students in year 1 to over 2500 in year 3. Our three industry partners have remained active throughout the project. During the third and final year in the classrooms, we focused on the sustainable aspects of the project. In particular, on how the intervention support has evolved each year based on data, support requests from the school divisions, and in scaffolding “ownership” of the engineering activities. Qualitative data were used to support our understanding of teachers’ confidence to incorporate engineering into their lessons plans and how their confidence changed over time. Noteworthy, our student data analysis resulted in an instrument change for the third year; however due to COVID, pre and post data was limited to schools who taught on a semester basis. Throughout the project we have utilized the ITEST STEM Workforce Education Helix model to support a pragmatic approach of our research informing our practice to enable an “iterative relationship between STEM content development and STEM career development activities… within the cultural context of schools, with teachers supported by professional development, and through programs supported by effective partnerships.” For example, over the course of the project, scaffolding from the University leading interventions to teachers leading interventions occurred.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.51355/jstem.2016.21
