More Like this

1. Generating Genre-Based Automatic Feedback on English for Research Publication Purposes

   https://doi.org/10.1558/cj.26273  

   Link, Stephanie; Redmon, Robert; Shamsi, Yaser; Hagan, Martin (March 2024, CALICO Journal)

   Smith, Bryan; Oskoz, A. (Ed.)

   Artificial intelligence (AI) for supporting second language (L2) writing processes and practices has garnered increasing interest in recent years, establishing AI-mediated L2 writing as a new norm for many multilingual classrooms. As such, the emergence of AI-mediated technologies has challenged L2 writing instructors and their philosophies regarding computer-assisted language learning (CALL) and teaching. Technologies that can combine principled pedagogical practices and the benefits of AI can help to change the landscape of L2 writing instruction while maintaining the integrity of knowledge production that is so important to CALL instructors. To align L2 instructional practices and CALL technologies, we discuss the development of an AI-mediated L2 writing technology that leverages genre-based instruction (GBI) and large language models to provide L2 writers and instructors with tools to enhance English for research publication purposes. Our work reports on the accuracy, precision, and recall of our network classification, which surpass previously reported research in the field of genre-based automated writing evaluation by offering a faster network training approach with higher accuracy of feedback provision and new beginnings for genre-based learning systems. Implications for tool development and GBI are discussed. 

   more » « less
2. Learning from Natural Language Feedback

   Chen, A; Scheurer, J; Campos, JA; Korbak, T; Chan, JS; Bowman, SR; Cho, K; Perez, E (February 2024, Transactions on machine learning research)

   The potential for pre-trained large language models (LLMs) to use natural language feedback at inference time has been an exciting recent development. We build upon this observation by formalizing an algorithm for learning from natural language feedback at training time instead, which we call Imitation learning from Language Feedback (ILF). ILF requires only a small amount of human-written feedback during training and does not require the same feedback at test time, making it both user-friendly and sample-efficient. We further show that ILF can be seen as a form of minimizing the KL divergence to the target distribution and demonstrate proof-of-concepts on text summarization and program synthesis tasks. For code generation, ILF improves a Codegen-Mono 6.1B model’s pass@1 rate from 22% to 36% on the MBPP benchmark, outperforming both fine-tuning on MBPP and on human- written repaired programs. For summarization, we show that ILF can be combined with learning from human preferences to improve a GPT-3 model’s summarization performance to be comparable to human quality, outperforming fine-tuning on human-written summaries. Overall, our results suggest that ILF is both more effective and sample-efficient than training exclusively on demonstrations for improving an LLM’s performance on a variety of tasks. 

   more » « less
3. An LLM-Guided Tutoring System for Social Skills Training

   https://doi.org/10.1609/aaai.v39i28.35353  

   Guevarra, Michael; Bhattacharjee, Indronil; Das, Srijita; Wayllace, Christabel; Demmans_Epp, Carrie; Taylor, Matthew E; Tay, Alan (April 2025, Proceedings of the AAAI Conference on Artificial Intelligence)

   Social skills training targets behaviors necessary for success in social interactions. However, traditional classroom training for such skills is often insufficient to teach effective communication — one-to-one interaction in real-world scenarios is preferred to lecture-style information delivery. This paper introduces a framework that allows instructors to collaborate with large language models to dynamically design realistic scenarios for students to communicate. Our framework uses these scenarios to enable student rehearsal, provide immediate feedback and visualize performance for both students and instructors. Unlike traditional intelligent tutoring systems, instructors can easily co-create scenarios with a large language model without technical skills. Additionally, the system generates new scenario branches in real time when existing options don't fit the student's response. 

   more » « less
4. Effect of Adaptive Communication Support on Human-AI Collaboration

   Liu, Shipeng; Shrutika, FNU; Zhang, Boshen; Huang, Zhehui; Qian, Feifei (November 2024, AAAI 2025 Workshop on Advancing LLM-Based Multi-Agent Collaboration)

   Effective human-AI collaboration requires agents to adopt their roles and levels of support based on human needs, task requirements, and complexity. Traditional human-AI teaming often relies on a pre-determined robot communication scheme, restricting teamwork adaptability in complex tasks. Leveraging the strong communication capabilities of Large Language Models (LLMs), we propose a Human-Robot Teaming Framework with Multi-Modal Language feedback (HRT-ML), a framework designed to enhance human-robot interaction by adjusting the frequency and content of language-based feedback. The HRT-ML framework includes two core modules: a Coordinator for high-level, low-frequency strategic guidance and a Manager for task-specific, high-frequency instructions, enabling passive and active interactions with human teammates. To assess the impact of language feedback in collaborative scenarios, we conducted experiments in an enhanced Overcooked-AI game environment with varying levels of task complexity (easy, medium, hard) and feedback frequency (inactive, passive, active, superactive). Our results show that as task complexity increases relative to human capabilities, human teammates exhibited stronger preferences toward robotic agents that can offer frequent, proactive support. However, when task complexities exceed the LLM's capacity, noisy and inaccurate feedback from superactive agents can instead hinder team performance, as it requires human teammates to increase their effort to interpret and respond to the large amount of communications, with limited performance return. Our results offer a general principle for robotic agents to dynamically adjust their levels and frequencies of communication to work seamlessly with humans and achieve improved teaming performance. 

   more » « less
5. Design and Validation of a Low-Cost Mobile EEG-Based Brain–Computer Interface

   https://doi.org/10.3390/s23135930  

   Craik, Alexander; González-España, Juan José; Alamir, Ayman; Edquilang, David; Wong, Sarah; Sánchez Rodríguez, Lianne; Feng, Jeff; Francisco, Gerard E.; Contreras-Vidal, Jose L. (July 2023, Sensors)

   Objective: We designed and validated a wireless, low-cost, easy-to-use, mobile, dry-electrode headset for scalp electroencephalography (EEG) recordings for closed-loop brain–computer (BCI) interface and internet-of-things (IoT) applications. Approach: The EEG-based BCI headset was designed from commercial off-the-shelf (COTS) components using a multi-pronged approach that balanced interoperability, cost, portability, usability, form factor, reliability, and closed-loop operation. Main Results: The adjustable headset was designed to accommodate 90% of the population. A patent-pending self-positioning dry electrode bracket allowed for vertical self-positioning while parting the user’s hair to ensure contact of the electrode with the scalp. In the current prototype, five EEG electrodes were incorporated in the electrode bracket spanning the sensorimotor cortices bilaterally, and three skin sensors were included to measure eye movement and blinks. An inertial measurement unit (IMU) provides monitoring of head movements. The EEG amplifier operates with 24-bit resolution up to 500 Hz sampling frequency and can communicate with other devices using 802.11 b/g/n WiFi. It has high signal–to–noise ratio (SNR) and common–mode rejection ratio (CMRR) (121 dB and 110 dB, respectively) and low input noise. In closed-loop BCI mode, the system can operate at 40 Hz, including real-time adaptive noise cancellation and 512 MB of processor memory. It supports LabVIEW as a backend coding language and JavaScript (JS), Cascading Style Sheets (CSS), and HyperText Markup Language (HTML) as front-end coding languages and includes training and optimization of support vector machine (SVM) neural classifiers. Extensive bench testing supports the technical specifications and human-subject pilot testing of a closed-loop BCI application to support upper-limb rehabilitation and provides proof-of-concept validation for the device’s use at both the clinic and at home. Significance: The usability, interoperability, portability, reliability, and programmability of the proposed wireless closed-loop BCI system provides a low-cost solution for BCI and neurorehabilitation research and IoT applications. 

   more » « less