skip to main content


Search for: All records

Creators/Authors contains: "Hutchins, Nicole"

Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

  1. This research explores a novel human-in-the-loop approach that goes beyond traditional prompt engineering approaches to harness Large Language Models (LLMs) with chain-of-thought prompting for grading middle school students’ short answer formative assessments in science and generating useful feedback. While recent efforts have successfully applied LLMs and generative AI to automatically grade assignments in secondary classrooms, the focus has primarily been on providing scores for mathematical and programming problems with little work targeting the generation of actionable insight from the student responses. This paper addresses these limitations by exploring a human-in-the-loop approach to make the process more intuitive and more effective. By incorporating the expertise of educators, this approach seeks to bridge the gap between automated assessment and meaningful educational support in the context of science education for middle school students. We have conducted a preliminary user study, which suggests that (1) co-created models improve the performance of formative feedback generation, and (2) educator insight can be integrated at multiple steps in the process to inform what goes into the model and what comes out. Our findings suggest that in-context learning and human-in-the-loop approaches may provide a scalable approach to automated grading, where the performance of the automated LLM-based grader continually improves over time, while also providing actionable feedback that can support students’ open-ended science learning. 
    more » « less
    Free, publicly-accessible full text available October 9, 2024
  2. Martin Fred ; Norouzi, Narges ; Rosenthal, Stephanie (Ed.)
    This paper examines the use of LLMs to support the grading and explanation of short-answer formative assessments in K12 science topics. While significant work has been done on programmatically scoring well-structured student assessments in math and computer science, many of these approaches produce a numerical score and stop short of providing teachers and students with explanations for the assigned scores. In this paper, we investigate few-shot, in-context learning with chain-of-thought reasoning and active learning using GPT-4 for automated assessment of students’ answers in a middle school Earth Science curriculum. Our findings from this human-in-the-loop approach demonstrate success in scoring formative assessment responses and in providing meaningful explanations for the assigned score. We then perform a systematic analysis of the advantages and limitations of our approach. This research provides insight into how we can use human-in-the-loop methods for the continual improvement of automated grading for open-ended science assessments. 
    more » « less
  3. Grieff, S. (Ed.)
    Recently there has been increased development of curriculum and tools that integrate computing (C) into Science, Technology, Engineering, and Math (STEM) learning environments. These environments serve as a catalyst for authentic collaborative problem-solving (CPS) and help students synergistically learn STEM+C content. In this work, we analyzed students’ collaborative problem-solving behaviors as they worked in pairs to construct computational models in kinematics. We leveraged social measures, such as equity and turn-taking, along with a domain-specific measure that quantifies the synergistic interleaving of science and computing concepts in the students’ dialogue to gain a deeper understanding of the relationship between students’ collaborative behaviors and their ability to complete a STEM+C computational modeling task. Our results extend past findings identifying the importance of synergistic dialogue and suggest that while equitable discourse is important for overall task success, fluctuations in equity and turn-taking at the segment level may not have an impact on segment-level task performance. To better understand students’ segment-level behaviors, we identified and characterized groups’ planning, enacting, and reflection behaviors along with monitoring processes they employed to check their progress as they constructed their models. Leveraging Markov Chain (MC) analysis, we identified differences in high- and low-performing groups’ transitions between these phases of students’ activities. We then compared the synergistic, turn-taking, and equity measures for these groups for each one of the MC model states to gain a deeper understanding of how these collaboration behaviors relate to their computational modeling performance. We believe that characterizing differences in collaborative problem-solving behaviors allows us to gain a better understanding of the difficulties students face as they work on their computational modeling tasks. 
    more » « less
    Free, publicly-accessible full text available September 15, 2024
  4. Abstract  
    more » « less
    Free, publicly-accessible full text available July 25, 2024
  5. null (Ed.)
    Strategies are an important component of self-regulated learning frameworks. However, the characterization of strategies in these frameworks is often incomplete: (1) they lack an operational definition of strategies; (2) there is limited understanding of how students develop and apply strategies; and (3) there is a dearth of systematic and generalizable approaches to measure and evaluate strategies when students’ work in open-ended learning environments (OELEs). This paper develops systematic methods for detecting, interpreting, and analyzing students’ use of strategies in OELEs, and demonstrates how students’ strategies evolve across tasks. We apply this framework in the context of tasks that students perform as they learn science topics by building conceptual and computational models in an OELE. Data from a classroom study, where sixth-grade students (N = 52) worked on science model-building activities in our Computational Thinking using Simulation and Modeling (CTSiM) environment demonstrates how we interpret students’ strategy use, and how strategy use relates to their learning performance. We also demonstrate how students’ strategies evolve as they work on multiple model-building tasks. The results demonstrate the effectiveness of our strategy framework in analyzing students’ behaviors and performance in CTSiM. 
    more » « less
  6. Careers in science, technology, engineering, and mathematics (STEM) increasingly rely on computational thinking (CT) to explore scientific processes and apply scientific knowledge to the solution of real-world problems. Integrating CT with science and engineering also helps broaden participation in computing for students who otherwise would not have access to CT learning. Using a set of emergent design guidelines for scaffolding integrated STEM and CT curricular experiences, we designed the Water Runoff Challenge (WRC) - a three-week unit that integrates Earth science, engineering, and CT. We implemented the WRC with 99 sixth grade students and analyzed students’ learning artifacts and pre/post assessments to characterize students’ learning process in the WRC. We use a vignette to illustrate how anchoring CT tasks to STEM contexts supported CT learning for a student with low prior CT proficiency. 
    more » « less
  7. C2STEM is a web-based learning environment founded on a novel paradigm that combines block-structured, visual programming with the concept of domain specific modeling languages (DSMLs) to promote the synergistic learning of discipline-specific and computational thinking (CT) concepts and practices. Our design-based, collaborative learning environment aims to provide students in K-12 classrooms with immersive experiences in CT through computational modeling in realistic scenarios (e.g., building models of scientific phenomena). The goal is to increase student engagement and include inclusive opportunities for developing key computational skills needed for the 21st century workforce. Research implementations that include a semester-long high school physics classroom study have demonstrated the effectiveness of our approach in supporting synergistic learning of STEM and CS/CT concepts and practices, especially when compared to a traditional classroom approach. This technology demonstration will showcase our CS+X (X = physics, marine biology, or earth science) learning environment and associated curricula. Participants can engage in our design process and learn how to develop curricular modules that cover STEM and CS/CT concepts and practices. Our work is supported by an NSF STEM+C grant and involves a multi-institutional team comprising Vanderbilt University, SRI International, Looking Glass Ventures, Stanford University, Salem State University, and ETR. More information, including example computational modeling tasks, can be found at C2STEM.org. 
    more » « less
  8. The paper presents RoboScape, a collaborative, networked robotics environment that makes key ideas in computer science accessible to groups of learners in informal learning spaces and K-12 classrooms. RoboScape is built on top of NetsBlox, an open-source, networked, visual programming environment based on Snap! that is specifically designed to introduce students to distributed computation and computer networking. RoboScape provides a twist on the state of the art of robotics learning platforms. First, a user's program controlling the robot runs in the browser and not on the robot. There is no need to download the program to the robot and hence, development and debugging become much easier. Second, the wireless communication between a student's program and the robot can be overheard by the programs of the other students. This makes cybersecurity an immediate need that students realize and can work to address. We have designed and delivered a cybersecurity summer camp to 24 students in grades between 7 and 12. The paper summarizes the technology behind RoboScape, the hands-on curriculum of the camp and the lessons learned. 
    more » « less