More Like this

1. Exploring LLM-Generated Feedback for Economics Essays: How Teaching Assistants Evaluate and Envision Its Use

   https://doi.org/10.1007/978-3-031-98417-4_28  

   Lu, Xinyi; Mahesh, Aditya; Shen, Zejia; Dudley, Mitchell; Sano, Larissa; Wang, Xu (July 2025, Springer Nature Switzerland)

   Cristea, Alexandra; Walker, Erin; Lu, Yu; Santos, Olga (Ed.)

   This project examines the prospect of using AI-generated feedback as suggestions to expedite and enhance human instructors’ feedback provision. In particular, we focus on understanding the teaching assistants’ perspectives on the quality of AI-generated feedback and how they may or may not utilize AI feedback in their own workflows. We situate our work in a foundational college Economics class, which has frequent short essay assignments. We developed an LLM-powered feedback engine that generates feedback on students’ essays based on grading rubrics used by the teaching assistants (TAs). To ensure that TAs can meaningfully critique and engage with the AI feedback, we had them complete their regular grading jobs. For a randomly selected set of essays that they had graded, we used our feedback engine to generate feedback and displayed the feedback as in-text comments in a Word document. We then performed think-aloud studies with 5 TAs over 20 1-hour sessions to have them evaluate the AI feedback, contrast the AI feedback with their handwritten feedback, and share how they envision using the AI feedback if they were offered as suggestions. The study highlights the importance of providing detailed rubrics for AI to generate high-quality feedback for knowledge-intensive essays. TAs considered that using AI feedback as suggestions during their grading could expedite grading, enhance consistency, and improve overall feedback quality. We discuss the importance of decomposing the feedback generation task into steps and presenting intermediate results, in order for TAs to use the AI feedback. 

   more » « less
2. Enhancing active learning through collaboration between human teachers and generative AI

   https://doi.org/10.1016/j.caeo.2024.100183  

   Pahi, Kritish; Hawlader, Shiplu; Hicks, Eric; Zaman, Alina; Phan, Vinhthuy (June 2024, Computers and Education Open)

   To address the increasing demand for AI literacy, we introduced a novel active learning approach that leverages both teaching assistants (TAs) and generative AI to provide feedback during in-class exercises. This method was evaluated through two studies in separate Computer Science courses, focusing on the roles and impacts of TAs in this learning environment, as well as their collaboration with ChatGPT in enhancing student feedback. The studies revealed that TAs were effective in accurately determining students’ progress and struggles, particularly in areas such as “backtracking”, where students faced significant challenges. This intervention’s success was evident from high student engagement and satisfaction levels, as reported in an end-of-semester survey. Further findings highlighted that while TAs provided detailed technical assessments and identified conceptual gaps effectively, ChatGPT excelled in presenting clarifying examples and offering motivational support. Despite some TAs’ resistance to fully embracing the feedback guidelines-specifically their reluctance to provide encouragement-the collaborative feedback process between TAs and ChatGPT improved the quality of feedback in several aspects, including technical accuracy and clarity in explaining conceptual issues. These results suggest that integrating human and artificial intelligence in educational settings can significantly enhance traditional teaching methods, creating a more dynamic and responsive learning environment. Future research will aim to improve both the quality and efficiency of feedback, capitalizing on unique strengths of both human and AI to further advance educational practices in the field of computing. 

   more » « less
3. Human-AI Collaboration in a Student Discussion Forum

   https://doi.org/10.1145/3640544.3645215  

   Laney, Mason; Dewan, Prasun (March 2024, ACM)

   The recent public releases of AI tools such as ChatGPT have forced computer science educators to reconsider how they teach. These tools have demonstrated considerable ability to generate code and answer conceptual questions, rendering them incredibly useful for completing CS coursework. While overreliance on AI tools could hinder students’ learning, we believe they have the potential to be a helpful resource for both students and instructors alike. We propose a novel system for instructor-mediated GPT interaction in a class discussion board. By automatically generating draft responses to student forum posts, GPT can help Teaching Assistants (TAs) respond to student questions in a more timely manner, giving students an avenue to receive fast, quality feedback on their solutions without turning to ChatGPT directly. Additionally, since they are involved in the process, instructors can ensure that the information students receive is accurate, and can provide students with incremental hints that encourage them to engage critically with the material, rather than just copying an AI-generated snippet of code. We utilize Piazza—a popular educational forum where TAs help students via text exchanges—as a venue for GPT-assisted TA responses to student questions. These student questions are sent to GPT-4 alongside assignment instructions and a customizable prompt, both of which are stored in editable instructor-only Piazza posts. We demonstrate an initial implementation of this system, and provide examples of student questions that highlight its benefits. 

   more » « less
4. Towards a Framework for Smart Classrooms that Teach Instructors to Teach

   Gerritsen, David; Zimmerman, John; Ogan, Amy (July 2018, International Conference of the Learning Sciences)

   Teaching Assistants (TAs) play a major role in higher education; however, they receive little if any training on how to teach. Quality training requires access to grounded feedback and relevant suggestions for improvement. We developed a framework for using features of a smart classroom. This work reframes the instructor as the learner. It provides training on discursive practices with feedback based on the instructor’s in-class behaviors. We built and deployed a system based on this framework to five STEM TAs as part of a larger study. This paper: discusses the action-reflection-planning framework we used, provides evidence for how the framework addresses TA learning goals, and discusses how other researchers might make use of the framework. 

   more » « less
5. The Ideal versus the Real Deal in Assessment of Physics Lab Report Writing

   https://doi.org/10.14738/aivp.112.14406  

   Passonneau, R; Koenig, K; Li, Z; Soddano, J (April 2023, European journal of applied sciences)

   James, C (Ed.)

   Effective writing is important for communicating science ideas, and for writing-to-learn in science. This paper investigates lab reports from a large-enrollment college physics course that integrates scientific reasoning and science writing. While analytic rubrics have been shown to define expectations more clearly for students, and to improve reliability of assessment, there has been little investigation of how well analytic rubrics serve students and instructors in large-enrollment science classes. Unsurprisingly, we found that grades administered by teaching assistants (TAs) do not correlate with reliable post-hoc assessments from trained raters. More important, we identified lost learning opportunities for students, and misinformation for instructors about students’ progress. We believe our methodology to achieve post-hoc reliability is straightforward enough to be used in classrooms. A key element is the development of finer-grained rubrics for grading that are aligned with the rubrics provided to students to define expectations, but which reduce subjectivity of judgements and grading time. We conclude that the use of dual rubrics, one to elicit independent reasoning from students and one to clarify grading criteria, could improve reliability and accountability of lab report assessment, which could in turn elevate the role of lab reports in the instruction of scientific inquiry. 

   more » « less