More Like this

1. Identifying Explanations Within Student-Tutor Chat Logs

   Prihar, Ethan; Moore, Alexander; Heffernan, Neil (July 2022, Proceedings of the 15th International Conference on Educational Data Mining, International Educational Data Mining Society)

   To improve student learning outcomes within online learning platforms, struggling students are often provided with on-demand supplemental instructional content. Recently, services like Yup (yup.com) and UPcheive (upchieve.org) have begun to offer on-demand live tutoring sessions with qualified educators, but the availability of tutors and the cost associated with hiring them prevents many students from having access to live support. To help struggling students and offset the inequities intrinsic to high-cost services, we are attempting to develop a process that uses large language representation models to algorithmically identify relevant support messages from these chat logs, and distribute them to all students struggling with the same content. In an empirical evaluation of our methodology we were able to identify messages from tutors to students struggling with middle school mathematics problems that qualified as explanations of the content. However, when we distributed these explanations to students outside of the tutoring sessions, they had an overall negative effect on the students’ learning. Moving forward, we want to be able to identify messages that will promote equity and have a positive impact on students. 

   more » « less
2. Undergraduate Computing Tutors' Perceptions of their Roles, Stressors, and Barriers to Effectiveness

   https://doi.org/10.1145/3641554.3701784  

   Villegas_Molina, Ismael; Kim, Jeannie; Montalvo, Audria; Larragoitia, Apollo; Lim, Rachel S; Guo, Philip J; Krause-Levy, Sophia; Porter, Leo (February 2025, ACM)

   Undergraduate teaching assistants (tutors) are commonly employed in computing courses to help students with programming assignments. Prior research in computing education has reported the benefits of tutoring both for students and for the tutors' own learning. In contrast, recent research that examined actual tutoring sessions has reported that these sessions may be less productive than one might hope, with tutors often just giving students the answers to their problems without trying to teach the underlying concepts. To better understand why tutors may be employing these suboptimal practices, we interviewed ten tutors across early computing courses in higher education to identify their perceived role in these sessions, what stressors and factors influence their ability to perform their job effectively, and what kinds of best practices they learned in their tutor training course. Tutors reported their roles around student learning, gauging student understanding, identifying or providing solutions to students, and providing socioemotional support. They reported their stressors around environmental factors (e.g., number of students waiting to be helped, preparation time, peer-tutor frustrations), internal influences, student behavior, student skill levels, and feeling the need to ''read a student's mind.'' Regarding their tutor training course, Tutors reported learning about interaction guidelines and procedures and question-based problem solving. We conclude by discussing how these results may contribute to the less-effective behaviors seen in prior research and potential ways to improve tutoring in computing courses. 

   more » « less
3. Comparing Different Approaches to Generating Mathematics Explanations Using Large Language Models

   https://doi.org/10.1007/978-3-031-36336-8_45  

   Prihar, Ethan; Lee, Morgan; Hopman, Mia; Kalai, Adam T.; Vempala, Sofia; Wang, Allison; Wickline, Gabriel; Murray, Aly; Heffernan, Neil (July 2023, AIED 2023)

   Large language models have recently been able to perform well in a wide variety of circumstances. In this work, we explore the possibility of large language models, specifically GPT-3, to write explanations for middle-school mathematics problems, with the goal of eventually using this process to rapidly generate explanations for the mathematics problems of new curricula as they emerge, shortening the time to integrate new curricula into online learning platforms. To generate explanations, two approaches were taken. The first approach attempted to summarize the salient advice in tutoring chat logs between students and live tutors. The second approach attempted to generate explanations using few-shot learning from explanations written by teachers for similar mathematics problems. After explanations were generated, a survey was used to compare their quality to that of explanations written by teachers. We test our methodology using the GPT-3 language model. Ultimately, the synthetic explanations were unable to outperform teacher written explanations. In the future more powerful large language models may be employed, and GPT-3 may still be effective as a tool to augment teachers’ process for writing explanations, rather than as a tool to replace them. The explanations, survey results, analysis code, and a dataset of tutoring chat logs are all available at https://osf.io/wh5n9/. 

   more » « less
4. Personalized Robot Tutoring using the Assistive Tutor POMDP (AT-POMDP)

   Ramachandran, A.; Sebo, S.; Scassellati, B. (January 2019, AAAI)

   Selecting appropriate tutoring help actions that account for both a student’s content mastery and engagement level is essential for effective human tutors, indicating the critical need for these skills in autonomous tutors. In this work, we formulate the robot-student tutoring help action selection problem as the Assistive Tutor partially observable Markov decision process (AT-POMDP). We designed the AT-POMDP and derived its parameters based on data from a prior robot-student tutoring study. The policy that results from solving the ATPOMDP allows a robot tutor to decide upon the optimal tutoring help action to give a student, while maintaining a belief of the student’s mastery of the material and engagement with the task. This approach is validated through a between-subjects field study, which involved 4th grade students (n = 28) interacting with a social robot solving long division problems over five sessions. Students who received help from a robot using the AT-POMDP policy demonstrated significantly greater learning gains than students who received help from a robot with a fixed help action selection policy. Our results demonstrate that this robust computational framework can be used effectively to deliver diverse and personalized tutoring support over time for students. 

   more » « less
5. Comparing Different Approaches to Generating Mathematics Explanations Using Large Language Models

   Prihar, E. (July 2023, Journal of artificial intelligence in education)

   Large language models have recently been able to perform well in a wide variety of circumstances. In this work, we explore the possi- bility of large language models, specifically GPT-3, to write explanations for middle-school mathematics problems, with the goal of eventually us- ing this process to rapidly generate explanations for the mathematics problems of new curricula as they emerge, shortening the time to inte- grate new curricula into online learning platforms. To generate expla- nations, two approaches were taken. The first approach attempted to summarize the salient advice in tutoring chat logs between students and live tutors. The second approach attempted to generate explanations us- ing few-shot learning from explanations written by teachers for similar mathematics problems. After explanations were generated, a survey was used to compare their quality to that of explanations written by teachers. We test our methodology using the GPT-3 language model. Ultimately, the synthetic explanations were unable to outperform teacher written explanations. In the future more powerful large language models may be employed, and GPT-3 may still be effective as a tool to augment teachers’ process for writing explanations, rather than as a tool to replace them. The prompts, explanations, survey results, analysis code, and a dataset of tutoring chat logs are all available at BLINDED FOR REVIEW. 

   more » « less