More Like this

1. Exploring LLM-Generated Feedback for Economics Essays: How Teaching Assistants Evaluate and Envision Its Use

   Lu, Xinyi; Mahesh, Aditya; Shen, Zejia; Dudley, Mitchell; Sano, Larissa; Wang, Xu (July 2025, Springer Nature Switzerland)

   Cristea, Alexandra; Walker, Erin; Lu, Yu; Santos, Olga (Ed.)

   This project examines the prospect of using AI-generated feedback as suggestions to expedite and enhance human instructors’ feedback provision. In particular, we focus on understanding the teaching assistants’ perspectives on the quality of AI-generated feedback and how they may or may not utilize AI feedback in their own workflows. We situate our work in a foundational college Economics class, which has frequent short essay assignments. We developed an LLM-powered feedback engine that generates feedback on students’ essays based on grading rubrics used by the teaching assistants (TAs). To ensure that TAs can meaningfully critique and engage with the AI feedback, we had them complete their regular grading jobs. For a randomly selected set of essays that they had graded, we used our feedback engine to generate feedback and displayed the feedback as in-text comments in a Word document. We then performed think-aloud studies with 5 TAs over 20 1-hour sessions to have them evaluate the AI feedback, contrast the AI feedback with their handwritten feedback, and share how they envision using the AI feedback if they were offered as suggestions. The study highlights the importance of providing detailed rubrics for AI to generate high-quality feedback for knowledge-intensive essays. TAs considered that using AI feedback as suggestions during their grading could expedite grading, enhance consistency, and improve overall feedback quality. We discuss the importance of decomposing the feedback generation task into steps and presenting intermediate results, in order for TAs to use the AI feedback. 

   more » « less
2. Exploring LLM-Generated Feedback for Economics Essays: How Teaching Assistants Evaluate and Envision Its Use

   https://doi.org/10.1007/978-3-031-98417-4_28  

   Lu, Xinyi; Mahesh, Aditya; Shen, Zejia; Dudley, Mitchell; Sano, Larissa; Wang, Xu (July 2025, Springer Nature Switzerland)

   Cristea, Alexandra; Walker, Erin; Lu, Yu; Santos, Olga (Ed.)

   This project examines the prospect of using AI-generated feedback as suggestions to expedite and enhance human instructors’ feedback provision. In particular, we focus on understanding the teaching assistants’ perspectives on the quality of AI-generated feedback and how they may or may not utilize AI feedback in their own workflows. We situate our work in a foundational college Economics class, which has frequent short essay assignments. We developed an LLM-powered feedback engine that generates feedback on students’ essays based on grading rubrics used by the teaching assistants (TAs). To ensure that TAs can meaningfully critique and engage with the AI feedback, we had them complete their regular grading jobs. For a randomly selected set of essays that they had graded, we used our feedback engine to generate feedback and displayed the feedback as in-text comments in a Word document. We then performed think-aloud studies with 5 TAs over 20 1-hour sessions to have them evaluate the AI feedback, contrast the AI feedback with their handwritten feedback, and share how they envision using the AI feedback if they were offered as suggestions. The study highlights the importance of providing detailed rubrics for AI to generate high-quality feedback for knowledge-intensive essays. TAs considered that using AI feedback as suggestions during their grading could expedite grading, enhance consistency, and improve overall feedback quality. We discuss the importance of decomposing the feedback generation task into steps and presenting intermediate results, in order for TAs to use the AI feedback. 

   more » « less
3. Performance Is Not Everything: Audio Feedback Preferred Over Visual Feedback for Grasping Task in Virtual Reality

   https://doi.org/10.1145/3424636.3426897  

   Canales, Ryan; Jörg, Sophie (October 2020, Motion, Interaction and Games)

   null (Ed.)

   In this work, we investigate the influence that audio and visual feedback have on a manipulation task in virtual reality (VR). Without the tactile feedback of a controller, grasping virtual objects using one’s hands can result in slower interactions because it may be unclear to the user that a grasp has occurred. Providing alternative feedback, such as visual or audio cues, may lead to faster and more precise interactions, but might also affect user preference and perceived ownership of the virtual hands. In this study, we test four feedback conditions for virtual grasping. Three of the conditions provide feedback for when a grasp or release occurs, either visual, audio, or both, and one provides no feedback for these occurrences. We analyze the effect each feedback condition has on interaction performance, measure their effect on the perceived ownership of the virtual hands, and gauge user preference. In an experiment, users perform a pick-and-place task with each feedback condition. We found that audio feedback for grasping is preferred over visual feedback even though it seems to decrease grasping performance, and found that there were little to no differences in ownership between our conditions. 

   more » « less
4. Public Peer Review Motivates Higher Quality Feedback

   Wang, X. (June 2018, International Conference of the Learning Sciences)

   The role of feedback in learning has been well researched, but in practice high quality feedback may be scarce, for example when the source of feedback is from peer learners. Nevertheless, peer feedback may be the main source of formative feedback available in some settings, such as in Massive Open Online Courses (MOOCs). A key part of the problem may be that students do not have sufficient incentive to offer their best feedback in settings where supervision is minimal. In this paper, we investigate whether students provide feedback of higher quality when it is done in a public setting rather than in a private setting. We report on an experimental study with 65 participants randomly assigned to a public feedback and a private feedback condition. We report the effect of the manipulation in terms of the quality of feedback offered as measured by a validated coding scheme, the subjective rating of the feedback, the effect on propensity to revise and success at increasing the quality of the writing. Limitations of the study and implications for practice are discussed. 

   more » « less
5. Iterative Relevance Feedback for Answer Passage Retrieval with Passage-Level Semantic Match

   https://doi.org/10.1007/978-3-030-15712-8_36  

   Bi, K. (April 2019, Proceedings of the European Conference on Information Retrieval (ECIR 19))

   Relevance feedback techniques assume that users provide relevance judgments for the top k (usually 10) documents and then re-rank using a new query model based on those judgments. Even though this is effective, there has been little research recently on this topic because requiring users to provide substantial feedback on a result list is impractical in a typical web search scenario. In new environments such as voice-based search with smart home devices, however, feedback about result quality can potentially be obtained during users' interactions with the system. Since there are severe limitations on the length and number of results that can be presented in a single interaction in this environment, the focus should move from browsing result lists to iterative retrieval and from retrieving documents to retrieving answers. In this paper, we study iterative relevance feedback techniques with a focus on retrieving answer passages. We first show that iterative feedback can be at least as effective as the top-k approach on standard TREC collections, and more effective on answer passage collections. We then propose an iterative feedback model for answer passages based on semantic similarity at passage level and show that it can produce significant improvements compared to both word-based iterative feedback models and those based on term-level semantic similarity. 

   more » « less