More Like this

1. Enhancing LLM-Based Short Answer Grading with Retrieval-Augmented Generation

   https://doi.org/10.5281/zenodo.15870304  

   Chu, Yucheng; He, Peng; Li, Hang; Han, Haoyu; Yang, Kaiqi; Xue, Yu; Li, Tingting; Krajcik, Joseph; Tang, Jiliang (July 2025, International Educational Data Mining Society)

   Mills, Caitlin; Alexandron, Giora; Taibi, Davide; Lo_Bosco, Giosuè; Paquette, Luc (Ed.)

   Short answer assessment is a vital component of science education, allowing evaluation of students' complex three-dimensional understanding. Large language models (LLMs) that possess human-like ability in linguistic tasks are increasingly popular in assisting human graders to reduce their workload. However, LLMs' limitations in domain knowledge restrict their understanding in task-specific requirements and hinder their ability to achieve satisfactory performance. Retrieval-augmented generation (RAG) emerges as a promising solution by enabling LLMs to access relevant domain-specific knowledge during assessment. In this work, we propose an adaptive RAG framework for automated grading that dynamically retrieves and incorporates domain-specific knowledge based on the question and student answer context. Our approach combines semantic search and curated educational sources to retrieve valuable reference materials. Experimental results in a science education dataset demonstrate that our system achieves an improvement in grading accuracy compared to baseline LLM approaches. The findings suggest that RAG-enhanced grading systems can serve as reliable support with efficient performance gains. 

   more » « less
2. A Theory of Adaptive Scaffolding for LLM-Based Pedagogical Agents

   https://doi.org/10.1609/aaai.v40i3.37154  

   Cohn, Clayton; Rayala, Surya; Srivastava, Namrata; Fonteles, Joyce Horn; Jain, Shruti; Luo, Xinying; Mereddy, Divya; Mohammed, Naveeduddin; Biswas, Gautam (March 2026, Proceedings of the AAAI Conference on Artificial Intelligence)

   Large language models (LLMs) present new opportunities for creating pedagogical agents that engage in meaningful dialogue to support student learning. However, current LLM systems used in classrooms often lack the solid theoretical foundations found in earlier intelligent tutoring systems. To bridge this gap, we propose a framework that combines Evidence-Centered Design with Social Cognitive Theory and Zone of Proximal Development for adaptive scaffolding in LLM-based agents focused on STEM+C learning. We instantiate this framework with Inquizzitor, an LLM-based formative assessment agent that integrates human-AI hybrid intelligence and provides feedback grounded in cognitive science principles. Our findings show that Inquizzitor delivers high-quality assessment and interaction aligned with core learning theories, offering effective guidance that students value. This research demonstrates the potential for theory-driven LLM integration in education, highlighting the ability of these systems to provide adaptive and principled instruction. 

   more » « less
3. Optimizing In-Context Demonstrations for LLM-Based Automated Grading

   Chu, Yucheng; Li, Hang; Yang, Kaiqi; Copur-Gencturk, Yasemin; Haudek, Kevin; Krajcik, Joseph; Shin, Namsoo; Tang, Jialing (July 2026, Lecture notes in computer science)

   Blanchard, EG; Chen, G; Chi, M; Isotani, S (Ed.)

   Automated assessment of open-ended student responses is a critical capability for scaling personalized feedback in education. While large language models (LLMs) have shown promise in grading tasks via in-context learning (ICL), their reliability is heavily dependent on the selection of few-shot exemplars and the construction of high-quality rationales. Standard retrieval methods typically select examples based on semantic similarity, which often fails to capture subtle decision boundaries required for rubric adherence. Furthermore, manually crafting the expert rationales needed to guide these models can be a significant bottleneck. To address these limitations, we introduce GUIDE (Grading Using Iteratively Designed Exemplars), a framework that reframes exemplar selection and refinement in automated grading as a boundaryfocused optimization problem. GUIDE operates on a continuous loop of selection and refinement, employing novel contrastive operators to identify “boundary pairs” that are semantically similar but possess different grades. We enhance exemplars by generating discriminative rationales that explicitly articulate why a response receives a specific score to the exclusion of adjacent grades. Extensive experiments across datasets in physics, chemistry, and pedagogical content knowledge demonstrate that GUIDE significantly outperforms standard retrieval baselines. By focusing the model’s attention on the precise edges of rubric, our approach shows exceptionally robust gains on borderline cases and improved rubric adherence. GUIDE paves the way for trusted, scalable assessment systems that align closely with human pedagogical standards. 

   more » « less
4. OpenTutor: Designing a Rapid-Authored Tutor that Learns as you Grade

   https://doi.org/10.32473/flairs.v34i1.128576  

   Nye, Benjamin D; Sanghrajka, Rushit; Bodhwani, Vinit; Acob, Martin; Budziwojski, Daniel; Carr, Kayla; Kirshner, Larry; Swartout, William R (May 2021, The International FLAIRS Conference Proceedings)

   Despite strong evidence that dialog-based intelligent tutoring systems (ITS) can increase learning gains, few courses include these tutors. In this research, we posit that existing dialog-based tutoring systems are not widely used because they are too complex and unfamiliar for a typical teacher to adapt or augment. OpenTutor is an open-source research project intended to scale up dialog-based tutoring by enabling ordinary teachers to rapidly author and improve dialog-based ITS, where authoring is presented through familiar tasks such as assessment item creation and grading. Formative usability results from a set of five non-CS educators are presented, which indicate that the OpenTutor system was relatively easy to use but that teachers would closely consider the cost benefit for time vs. student outcomes. Specifically, while OpenTutor grading was faster than expected, teachers reported that they would only spend any additional time (compared to a multiple choice) if the content required deeper learning. To decrease time to train answer classifiers, OpenTutor is investigating ways to reduce cold-start problems for tutoring dialogs. 

   more » « less
5. Hao Fayin: Developing Automated Audio Assessment Tools for a Chinese Language Course

   Varatharaj, A.; Botelho, A. F.; Lu, X.; Heffernan, N. T. (July 2019, The Twelfth International Conference on Educational Data Mining)

   We present and evaluate a machine learning based system that automatically grades audios of students speaking a foreign language. The use of automated systems to aid the assessment of student performance holds great promise in augmenting the teacher’s ability to provide meaningful feedback and instruction to students. Teachers spend a significant amount of time grading student work and the use of these tools can save teachers a significant amount of time on their grading. This additional time could be used to give personalized attention to each student. Significant prior research has focused on the grading of closed-form problems, open-ended essays and textual content. However, little research has focused on audio content that is much more prevalent in the language-study education. In this paper, we explore the development of automated assessment tools for audio responses in a college-level Chinese language-learning course. We analyze several challenges faced while working with data of this type as well as the generation and extraction of features for the purpose of building machine learning models to aid in the assessment of student language learning. 

   more » « less