More Like this

1. Design and Evaluation Attributes for Scalable, Cost-Effective Personalization of LLM Tutors in Programming Education

   Rai, A; Chen, L; Breazeal, C; Ramesh, B; Long, Y; Aria, A (December 2024, International Conference on Information Systems 2024)

   This paper examines the design and evaluation of Large Language Model (LLM) tutors for Python programming, focusing on personalization that accommodates diverse student backgrounds. It highlights the challenges faced by socioeconomically disadvantaged students in computing courses and proposes LLM tutors as a solution to provide inclusive educational support. The study explores two LLM tutors, Khanmigo and CS50.ai, assessing their ability to offer personalized learning experiences. By employing a focus group methodology at a public minority-serving institution, the research evaluates how these tutors meet varied educational goals and adapt to students’ diverse needs. The findings underscore the importance of advanced techniques to tailor interactions and integrate programming tools based on students' progress. This research contributes to the understanding of educational technologies in computing education and provides insights into the design and implementation of LLM tutors that effectively support equitable student success. 

   more » « less
2. Understanding Cross-lingual Pragmatic Misunderstandings in Email Communication

   https://doi.org/10.1145/3512976  

   Lim, Hajin; Cosley, Dan; Fussell, Susan R. (January 2022, Proceedings of the ACM on humancomputer interaction)

   Communication tools such as email facilitate communication and collaboration between speakers of different languages, who use two primary strategies—English as a common language and machine translation (MT) tools—to help them overcome language barriers. However, each of these communication strategies creates its own challenges for cross-lingual communication. In this paper, we compare how people’s interpretations of an email sender’s social intention, and their evaluation of the email and the senders, differ when using a common language versus MT in email communication. We conducted an online experiment in which monolingual native English speakers read and rated request emails written by native English speakers, emails written by bilingual Chinese speakers in English, and emails written in Chinese then machine-translated into English. We found that participants interpreted the social intentions of the email sender less accurately for machine-translated emails than for emails written by non-native speakers in English. Participants also rated the senders and emails less positively overall for machine-translated emails compared to emails written by non-native speakers in English. Based on these findings, we suggest design possibilities that could better aid multilingual communication. 

   more » « less
3. Large language models for whole-learner support: opportunities and challenges

   https://doi.org/10.3389/frai.2024.1460364  

   Mannekote, Amogh; Davies, Adam; Pinto, Juan D; Zhang, Shan; Olds, Daniel; Schroeder, Noah L; Lehman, Blair; Zapata-Rivera, Diego; Zhai, ChengXiang (October 2024, Frontiers in Artificial Intelligence)

   In recent years, large language models (LLMs) have seen rapid advancement and adoption, and are increasingly being used in educational contexts. In this perspective article, we explore the open challenge of leveraging LLMs to create personalized learning environments that support the “whole learner” by modeling and adapting to both cognitive and non-cognitive characteristics. We identify three key challenges toward this vision: (1) improving the interpretability of LLMs' representations of whole learners, (2) implementing adaptive technologies that can leverage such representations to provide tailored pedagogical support, and (3) authoring and evaluating LLM-based educational agents. For interpretability, we discuss approaches for explaining LLM behaviors in terms of their internal representations of learners; for adaptation, we examine how LLMs can be used to provide context-aware feedback and scaffold non-cognitive skills through natural language interactions; and for authoring, we highlight the opportunities and challenges involved in using natural language instructions to specify behaviors of educational agents. Addressing these challenges will enable personalized AI tutors that can enhance learning by accounting for each student's unique background, abilities, motivations, and socioemotional needs. 

   more » « less
4. Using Trained Tutors to Improve Mechanical Engineering Technology Student Writing

   Clippinger, D.; Pflueger, R. C.; Nozaki, S.; Bodenhamer, J. (June 2023, American Society for Engineering Education)

   In an earlier work, the authors compared the writing style of Mechanical Engineering Technology (MET) students in an “untutored” state to the writing style of “tutored” students, where the tutoring was provided by “generic” writing center tutors. The results of this study showed that aside from changes in the diction of the students’ work, there was little measurable improvement in the quality of student writing as measured by both the AAC&U VALUE Rubric and by the authors’ voice-development-style-diction method. The current work builds on the results of the previous work by providing training on a just-in-time basis for the writing center tutors. As with previous years, the students participating in the study were MET students in a last-semester capstone industrial design course. This course is organized around a series of open-ended industry-sponsored projects for which the students are expected to develop a solution to a mechanical engineering problem. The students work on the projects in teams of three or four students and complete the work over a two-semester sequence offered annually on a fall-spring basis. The assignment in the study was identical to that of previous years: an “analysis” report in which students are expected to apply content from previous courses to one aspect of the industry-sponsored design project. The present study will compare the results from three iterations of the study: the work of “untutored” students, i.e. those who did not received any writing center assistance whatsoever, those who tutored by “generic” writing center tutors, and lastly, the works of those tutored by tutors specifically trained in support of the specific intervention. In the two cases where tutor interaction occurred, it was required as a component of the course to ensure participation by the entire student cohort. In general, the interactions with the specially-trained tutors produced works with a more mature writing style on the part of the student as compared to those works produced by students who had interacted with the untrained tutors or no tutors at all. The work will also discuss survey data collected on the “generic” and specially-trained tutoring sessions and discuss the differences in the results. Preliminary results show that the specially-trained tutors reported pronounced levels of engagement in the tutoring session, as measured by student note-taking, student questions, student receptiveness to suggestions, and student desire to understand the reasoning behind the tutors’ suggestions. Specially-trained tutors also reported significantly higher levels of student interest suggestions about grammar, style, content, format, and citations. Overall, it is concluded that specific training for the tutors was most associated with increased levels of interaction between tutor and student. As the students in the final group (“trained tutors”) were told prior to the tutoring session that the tutors were “specially trained,” it remains to be determined if the increased interaction was due to better tutor preparation or a higher estimation of the value of the tutoring session on the part of the students receiving the tutoring. This is proposed as an extension to the current work. 

   more » « less
5. Using Trained Tutors to Improve Mechanical Engineering Technology Student Writing

   Clippinger, D. (June 2023, 2023 ASEE Annual Conference, Baltimore, MD, USA)

   In an earlier work, the authors compared the writing style of Mechanical Engineering Technology (MET) students in an “untutored” state to the writing style of “tutored” students, where the tutoring was provided by “generic” writing center tutors. The results of this study showed that aside from changes in the diction of the students’ work, there was little measurable improvement in the quality of student writing as measured by both the AAC&U VALUE Rubric and by the authors’ voice-development-style-diction method. The current work builds on the results of the previous work by providing training on a just-in-time basis for the writing center tutors. As with previous years, the students participating in the study were MET students in a last-semester capstone industrial design course. This course is organized around a series of open-ended industry-sponsored projects for which the students are expected to develop a solution to a mechanical engineering problem. The students work on the projects in teams of three or four students and complete the work over a two-semester sequence offered annually on a fall-spring basis. The assignment in the study was identical to that of previous years: an “analysis” report in which students are expected to apply content from previous courses to one aspect of the industry-sponsored design project. The present study will compare the results from three iterations of the study: the work of “untutored” students, i.e. those who did not received any writing center assistance whatsoever, those who tutored by “generic” writing center tutors, and lastly, the works of those tutored by tutors specifically trained in support of the specific intervention. In the two cases where tutor interaction occurred, it was required as a component of the course to ensure participation by the entire student cohort. In general, the interactions with the specially-trained tutors produced works with a more mature writing style on the part of the student as compared to those works produced by students who had interacted with the untrained tutors or no tutors at all. The work will also discuss survey data collected on the “generic” and specially-trained tutoring sessions and discuss the differences in the results. Preliminary results show that the specially-trained tutors reported pronounced levels of engagement in the tutoring session, as measured by student note-taking, student questions, student receptiveness to suggestions, and student desire to understand the reasoning behind the tutors’ suggestions. Specially-trained tutors also reported significantly higher levels of student interest suggestions about grammar, style, content, format, and citations. Overall, it is concluded that specific training for the tutors was most associated with increased levels of interaction between tutor and student. As the students in the final group (“trained tutors”) were told prior to the tutoring session that the tutors were “specially trained,” it remains to be determined if the increased interaction was due to better tutor preparation or a higher estimation of the value of the tutoring session on the part of the students receiving the tutoring. This is proposed as an extension to the current work. 

   more » « less