Search for: All records

Automated assessment of open-ended student responses is a critical capability for scaling personalized feedback in education. While large language models (LLMs) have shown promise in grading tasks via in-context learning (ICL), their reliability is heavily dependent on the selection of few-shot exemplars and the construction of high-quality rationales. Standard retrieval methods typically select examples based on semantic similarity, which often fails to capture subtle decision boundaries required for rubric adherence. Furthermore, manually crafting the expert rationales needed to guide these models can be a significant bottleneck. To address these limitations, we introduce GUIDE (Grading Using Iteratively Designed Exemplars), a framework that reframes exemplar selection and refinement in automated grading as a boundaryfocused optimization problem. GUIDE operates on a continuous loop of selection and refinement, employing novel contrastive operators to identify “boundary pairs” that are semantically similar but possess different grades. We enhance exemplars by generating discriminative rationales that explicitly articulate why a response receives a specific score to the exclusion of adjacent grades. Extensive experiments across datasets in physics, chemistry, and pedagogical content knowledge demonstrate that GUIDE significantly outperforms standard retrieval baselines. By focusing the model’s attention on the precise edges of rubric, our approach shows exceptionally robust gains on borderline cases and improved rubric adherence. GUIDE paves the way for trusted, scalable assessment systems that align closely with human pedagogical standards. 

Short answer assessment is a vital component of science education, allowing evaluation of students' complex three-dimensional understanding. Large language models (LLMs) that possess human-like ability in linguistic tasks are increasingly popular in assisting human graders to reduce their workload. However, LLMs' limitations in domain knowledge restrict their understanding in task-specific requirements and hinder their ability to achieve satisfactory performance. Retrieval-augmented generation (RAG) emerges as a promising solution by enabling LLMs to access relevant domain-specific knowledge during assessment. In this work, we propose an adaptive RAG framework for automated grading that dynamically retrieves and incorporates domain-specific knowledge based on the question and student answer context. Our approach combines semantic search and curated educational sources to retrieve valuable reference materials. Experimental results in a science education dataset demonstrate that our system achieves an improvement in grading accuracy compared to baseline LLM approaches. The findings suggest that RAG-enhanced grading systems can serve as reliable support with efficient performance gains. 

Open-text responses provide researchers and educators with rich, nuanced insights that multiple-choice questions cannot capture. When reliably assessed, such responses have the potential to enhance teaching and learning. However, scaling and consistently capturing these nuances remain significant challenges, limiting the widespread use of open-text questions in educational research and assessments. In this paper, we introduce and evaluate GradeOpt, a unified multiagent automatic short-answer grading (ASAG) framework that leverages large language models (LLMs) as graders for short-answer responses. More importantly, GradeOpt incorporates two additional LLM-based agents—the reflector and the refiner—into the multi-agent system. This enables GradeOpt to automatically optimize the original grading guidelines by performing self-reflection on its errors. To assess GradeOpt's effectiveness, we conducted experiments on two representative ASAG datasets, which include items designed to capture key aspects of teachers' pedagogical knowledge and students' learning progress. Our results demonstrate that GradeOpt consistently outperforms representative baselines in both grading accuracy and alignment with human evaluators across different knowledge domains. Finally, comprehensive ablation studies validate the contributions of GradeOpt's individual components, confirming their impact on overall performance. 

The 1988 Yellowstone fire altered the structure of the local forest ecosystem and left large non-recovery areas. This study assessed the pre-fire drivers and post-fire characteristics of the recovery and non-recovery areas and examined possible reasons driving non-recovery of the areas post-fire disturbance. Non-recovery and recovery areas were sampled with 44,629 points and 77,501 points, from which attribute values related to topography, climate, and subsequent soil conditions were extracted. We calculated the 1988 Yellowstone fire burn thresholds using the differenced Normalized Burn Ratio (dNBR) and official fire maps. We used a burn severity map from the US Forest Service to calculate the burn severity values. Spatial regressions and Chi-Square tests were applied to determine the statistically significant characteristics of a lack of recovery. The non-recovery areas were found to cover 1005.25 km2. Among 11 variables considered as potential factors driving recovery areas and 13 variables driving non-recovery areas, elevation and maximum temperature were found to have high Variance Inflation Factors (4.73 and 4.72). The results showed that non-recovery areas all experienced severe burns and were located at areas with steeper slopes (13.99°), more precipitation (871.73 mm), higher pre-fire vegetation density (NDVI = 0.38), higher bulk density (750.03 kg/m3), lower soil organic matter (165.61 g/kg), and lower total nitrogen (60.97 mg/L). Chi-square analyses revealed statistically different pre-fire forest species (p < 0.01) and soil order (p < 0.01) in the recovery and non-recovery areas. Although Inceptisols dominated in both recovery and non-recovery areas, however, the composition of Mollisols was higher in the non-recovery areas (14%) compared to the recovery areas (11%). This indicated the ecological memory of the non-recovery site reverting to grassland post-disturbance. Unlike conventional studies only focusing on recovery areas, this study analyzed the non-recovery areas and found the key characteristics that make a landscape not resilient to the 1988 Yellowstone fire. The significant effects of elevation, precipitation, and soil pH on recovery may be significant to the forest management and forest resilience in the post-fire period.