More Like this

1. A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students’ Formative Assessment Responses in Science

   Cohn, Clayton; Hutchins, Nicole; Biswas, Gautam (September 2023, AAAI Press)

   Martin Fred; Norouzi, Narges; Rosenthal, Stephanie (Ed.)

   This paper examines the use of LLMs to support the grading and explanation of short-answer formative assessments in K12 science topics. While significant work has been done on programmatically scoring well-structured student assessments in math and computer science, many of these approaches produce a numerical score and stop short of providing teachers and students with explanations for the assigned scores. In this paper, we investigate few-shot, in-context learning with chain-of-thought reasoning and active learning using GPT-4 for automated assessment of students’ answers in a middle school Earth Science curriculum. Our findings from this human-in-the-loop approach demonstrate success in scoring formative assessment responses and in providing meaningful explanations for the assigned score. We then perform a systematic analysis of the advantages and limitations of our approach. This research provides insight into how we can use human-in-the-loop methods for the continual improvement of automated grading for open-ended science assessments. 

   more » « less
2. A Principled Approach to Designing Assessments That Integrate Science and Computational Thinking

   https://doi.org/10.22318/cscl2018.384  

   Basu, S.; McElhaney, K. W.; Grover, S.; Harris, C. J.; Biswas, G. (July 2018, 13th International Conference of the Learning Sciences (ICLS))

   There is increasing interest in broadening participation in computational thinking (CT) by integrating CT into pre-college STEM curricula and instruction. Science, in particular, is emerging as an important discipline to support integrated learning. This highlights the need for carefully designed assessments targeting the integration of science and CT to help teachers and researchers gauge students’ proficiency with integrating the disciplines. We describe a principled design process to develop assessment tasks and rubrics that integrate concepts and practices across science, CT, and computational modeling. We conducted a pilot study with 10 high school students who responded to integrative assessment tasks as part of a physics-based computational modeling unit. Our findings indicate that the tasks and rubrics successfully elicit both Physics and CT constructs while distinguishing important aspects of proficiency related to the two disciplines. This work illustrates the promise of using such assessments formatively in integrated STEM and computing learning contexts. 

   more » « less
3. Teacher Preparation and Student Participation in a GIS-Focused Computer Science Unit Impact Students’ Geospatial Thinking

   Pagano, Lauren C; Sotelo, Jose; McGee-Tekula, Randi; McGee, Steven; Kolvoord, Robert A; Uttal, David H (April 2024, American Educational Research Association)

   Experience with geographic information systems (GIS) can improve students’ spatial skills and provide a foundation for success in STEM (Jant et al., 2019). Researchers and educators co-designed a GIS unit in which high school students learned to use ArcGIS software by exploring geospatial patterns in their local communities. Across three teachers, 134 students participated in the unit and completed a geospatial problem-solving assessment. Students’ performance on the assessment significantly increased from pre- to post-test. Students whose teachers had more GIS experience and completed graded GIS assessments scored higher on geospatial assessments and used more spatial language than students whose teachers had less GIS experience and graded on participation. Students’ expectancy, value, and cost of computer science varied across teachers, and may be linked to students’ ability to devote time to mapbuilding and their engagement with a GIS careers guide. We discuss the impacts of teacher training and lesson implementation on students’ geospatial thinking. 

   more » « less
4. Exploring Middle School Science Teachers’ Curriculum Sensemaking

   https://doi.org/10.1080/1046560X.2025.2497188  

   Wright, Gary; Zangori, Laura; Otto, Suzanne; Snyder, Rebekah; Cole, Laura (May 2025, Journal of Science Teacher Education)

   Borowczak, A; Dare, E; Tofel-Grehl, C (Ed.)

   A teacher’s working context is an important factor in how they make sense of and enact curriculum. Understanding how external factors (e.g. state and/or district policies, school cultural norms) interplay with teachers’ personal resources (e.g. self-understanding, rules of thumb for decision-making) can help identify supports for implementation of increasingly available standard aligned curriculum materials. However, in science education, limited research has explored how curriculum enactments are influenced by this complex interplay. In this qualitative embedded case study, we investigated how four middle school science teachers within the same school district used their internal resources to make sense of external factors when enacting new NGSS-aligned place-based curriculum materials. Data collection occurred over multiple years and included semi-structured individual and focus group interviews, lesson plans, weekly surveys, observations, and memos. Using thematic analysis, we found that a new district-level policy implementing a 6-week science assessment caused differential enactments of the unit, depending on which internal resources teachers drew on to make sense of the curriculum materials. Our findings contribute to further understanding how internal personal resources and external factors support and impede science teachers’ use of curriculum materials in ways that align, or do not align, with recent reform-based learning outlined in the NGSS. 

   more » « less
5. Growth in STEM Teachers’ Formative Assessment Practices as Teachers Remain in High-Need Districts

   Abramovitch, S.; Sevian, H. (April 2021, NARST 94th Annual International Conference)

   null (Ed.)

   The way teachers design activities and interact with students during instruction directly impacts students' opportunities to learn (OTL). Previous research has shown the merits of formative assessment (FA) in supporting students' active sense-making, leading to improved student success outcomes. We examined 22 STEM teachers' classroom videos, performing FA in class, taken from two different years of teaching in high-need districts. We then coded the videos according to two basic teaching moves - eliciting information about students' thinking or advanced learning. These moves can also be categorized as more authoritative (univocal) class discourse or more dialogic (multivocal). Our results show that thirteen out of the twenty-two teachers in this study diversified their teaching moves over time as they gained experience while persisting in high-need districts. The results also suggest five different teachers' clusters, representing different changes over time in these teachers' teaching moves. Teachers' reflections on challenges they faced while teaching and changes in their assessment practices over time suggest that changes in their teaching purposes resulted in shifting their teaching moves. These shifts supported their students' different challenges, building meaningful relationships with their students, and allowing them more OTL. 

   more » « less