More Like this

1. A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students’ Formative Assessment Responses in Science

   Cohn, Clayton ; Hutchins, Nicole ; Biswas, Gautam ( September 2023 , AAAI Press)

   Martin Fred ; Norouzi, Narges ; Rosenthal, Stephanie (Ed.)

   This paper examines the use of LLMs to support the grading and explanation of short-answer formative assessments in K12 science topics. While significant work has been done on programmatically scoring well-structured student assessments in math and computer science, many of these approaches produce a numerical score and stop short of providing teachers and students with explanations for the assigned scores. In this paper, we investigate few-shot, in-context learning with chain-of-thought reasoning and active learning using GPT-4 for automated assessment of students’ answers in a middle school Earth Science curriculum. Our findings from this human-in-the-loop approach demonstrate success in scoring formative assessment responses and in providing meaningful explanations for the assigned score. We then perform a systematic analysis of the advantages and limitations of our approach. This research provides insight into how we can use human-in-the-loop methods for the continual improvement of automated grading for open-ended science assessments. 

   more » « less
2. Improving Science Achievement—Is It Possible? Evaluating the Efficacy of a High School Chemistry and Physics Project-Based Learning Intervention

   https://doi.org/10.3102/0013189X211067742  

   Schneider, Barbara ; Krajcik, Joseph ; Lavonen, Jari ; Salmela-Aro, Katariina ; Klager, Christopher ; Bradford, Lydia ; Chen, I-Chien ; Baker, Quinton ; Touitou, Israel ; Peek-Brown, Deborah ; et al ( January 2022 , Educational Researcher)

   Crafting Engaging Science Environments is a high school chemistry and physics project-based learning intervention that meets Next Generation Science Standards performance expectations. It was administered to a diverse group of over 4,000 students in a randomized control trial in California and Michigan. Results show that treatment students, on average, performed 0.20 standard deviations higher than control students on an independently developed summative science assessment. Mediation analyses show an indirect path between teacher- and student-reported participation in modeling practices and science achievement. Exploratory analyses indicate positive treatment effects for enhancing college ambitions. Overall, results show that improving secondary school science learning is achievable with a coherent system comprising teacher and student learning experiences, professional learning, and formative unit assessments that support students in “doing” science.

    

   more » « less
3. Predicting student science achievement using post‐unit assessment performances in a coherent high school chemistry project‐based learning system

   https://doi.org/10.1002/tea.21815  

   He, Peng ; Chen, I‐Chien ; Touitou, Israel ; Bartz, Kayla ; Schneider, Barbara ; Krajcik, Joseph ( September 2022 , Journal of Research in Science Teaching)

   Student science proficiency development demands sustainable and coherent learning environment support. Scholars argue that project‐based learning (PBL) is an efficient approach to promoting student science learning, compared to conventional instructions. Yet, few studies have delved into the learning process to explore how a coherent PBL system consisting of curriculum, instruction, assessment, and professional learning promotes student learning. To address the gap, this study investigated whether students' science proficiency on the three post‐unit assessments predicted their achievement on a third‐party‐designed end‐of‐year summative science test in a coherent high school chemistry PBL system aligned with the recent US science standards. The study employed a cluster randomized experimental design to test an intervention using our PBL system and only used data from the treatment group. The sample consisted of 1344 treatment students who participated in our PBL intervention and underwent the pretest and end‐of‐year summative test. Students' responses to the three post‐unit assessments were selected and rated to indicate their science proficiency. Two‐level hierarchical linear models were employed to explore the effects of students' performances of three post‐unit assessments on their end‐of‐year summative achievement, considering and controlling for student prior knowledge (i.e., pretest and prior post‐unit assessments). This study suggests two main findings. First, students' science proficiency in the three units could cumulatively and individually predict their summative science achievement. Second, students' performances on the two types of tasks (i.e., developing and using models) in the three post‐unit assessments could also be used to predict their summative science achievement. This research contributes to the field by showing that a coherent standards‐aligned PBL system can significantly and sustainably impact student science proficiency development.

    

   more » « less
4. Growth in STEM Teachers’ Formative Assessment Practices as Teachers Remain in High-Need Districts

   Abramovitch, S. ; Sevian, H. ( April 2021 , NARST 94th Annual International Conference)

   null (Ed.)

   The way teachers design activities and interact with students during instruction directly impacts students' opportunities to learn (OTL). Previous research has shown the merits of formative assessment (FA) in supporting students' active sense-making, leading to improved student success outcomes. We examined 22 STEM teachers' classroom videos, performing FA in class, taken from two different years of teaching in high-need districts. We then coded the videos according to two basic teaching moves - eliciting information about students' thinking or advanced learning. These moves can also be categorized as more authoritative (univocal) class discourse or more dialogic (multivocal). Our results show that thirteen out of the twenty-two teachers in this study diversified their teaching moves over time as they gained experience while persisting in high-need districts. The results also suggest five different teachers' clusters, representing different changes over time in these teachers' teaching moves. Teachers' reflections on challenges they faced while teaching and changes in their assessment practices over time suggest that changes in their teaching purposes resulted in shifting their teaching moves. These shifts supported their students' different challenges, building meaningful relationships with their students, and allowing them more OTL. 

   more » « less
5. Supporting Teachers to Customize Curriculum for Self-Directed Learning

   https://doi.org/10.1007/s10956-022-09985-w  

   Gerard, Libby ; Bradford, Allison ; Linn, Marcia C. ( August 2022 , Journal of Science Education and Technology)

   Guiding teachers to customize curriculum has shown to improve science instruction when guided effectively. We explore how teachers use student data to customize a web-based science unit on plate tectonics. We study the implications for teacher learning along with the impact on student self-directed learning. During a professional development workshop, four 7th grade teachers reviewed logs of their students’ explanations and revisions. They used a curriculum visualization tool that revealed the pedagogy behind the unit to plan their customizations. To promote self-directed learning, the teachers decided to customize the guidance for explanation revision by giving students a choice among guidance options. They took advantage of the web-based unit to randomly assign students (N = 479) to either a guidance Choice or a no-choice condition. We analyzed logged student explanation revisions on embedded and pre-test/post-test assessments and teacher and student written reflections and interviews. Students in the guidance Choice condition reported that the guidance was more useful than those in the no-choice condition and made more progress on their revisions. Teachers valued the opportunity to review student work, use the visualization tool to align their customization with the knowledge integration pedagogy, and investigate the choice option empirically. These findings suggest that the teachers’ decision to offer choice among guidance options promoted aspects of self-directed learning.

    

   more » « less