More Like this

1. Comparing alternative approaches to debriefing in a tool to support peer-led simulation-based training

   https://doi.org/10.1007/978-3-031-09680-8_8  

   Katz, S.; Albacete, P.; Gallagher, J.; Jordan, P.; Platt, T.; Silliman, S.; Yang, T. (January 2022, Intelligent Tutoring Systems: 18th International Conference, ITS 2022)

   This poster describes an early-stage project. It introduces MedDbriefer, a tablet-based tool that allows small groups of paramedic students to practice realistic prehospital emergency care scenarios. While two or more students collaborate as members of an emergency medical service (EMS) team, a peer uses the tablet’s checklists to record the team’s actions. The system then analyzes the event log to provide an automated debriefing on the team’s performance. Although debriefing is purported to be one of simulation-based training’s most critical components, there is little research to guide human and automated debriefing. We are imple-menting two approaches to automated debriefing and will compare their effective-ness in an upcoming randomized controlled trial. 

   more » « less
2. ACT360: An Efficient 360-Degree Action Detection and Summarization Framework for Mission-Critical Training and Debriefing

   https://doi.org/10.1109/SMARTCOMP65954.2025.00053  

   Tiwari, Aditi; Nahrstedt, Klara (June 2025, IEEE)

   Effective training and debriefing are critical in high-stakes, mission-critical environments such as firefighting, where precision and error minimization are paramount. The traditional post-training analysis relies on the manual review of 2D video, a process that is time-consuming and lacks comprehensive situational awareness. To address these limitations, we introduce ACT360, a novel system that leverages 360-degree video and machine learning for automated action detection and efficient debriefing. ACT360 incorporates 360YOWO, a customized You Only Watch Once (YOWO) model enhanced with a spatial attention mechanism and equirectangular-aware convolution (EAC) to handle the unique distortions of panoramic video data. To enable deployment in resource-constrained environments, we apply quantization and model pruning, reducing the model size by 74% while maintaining robust accuracy (mAP drop of only 1.5 %, from 0.865 to 0.850) and improving inference speed. We validate our approach on a new, publicly available dataset of 55 labeled 360-degree videos covering seven key firefighting actions, recorded across various real-world practice sessions and environmental conditions. Furthermore, we integrate the pipeline with 360AIE (Action Insight Explorer), a web-based interface that provides automatic action detection, retrieval, and textual summarization of key events using large language models (LLMs), significantly improving post-incident analysis efficiency. ACT360 serves as a generalized framework for mission-critical debriefing, incorporating techniques such as EAC, spatial attention, summarization, and model optimization. These innovations apply to any training environment requiring lightweight action detection and structured nost-exercise analysis. 

   more » « less
3. Towards Supporting Technical and Non-Technical Skills Development by Using Multimodal Debriefing System After Multi-User VR-Based Simulation Training

   https://doi.org/10.22318/cscl2023.644173  

   Popov, Vitaliy; Li, Yaxuan; Cooke, James; Sample, Alanson; Cole, Michael. (June 2023, Computersupported collaborative learning)

   This NSF-funded study aims to develop and evaluate a novel debriefing system that aims to capture and visualize multimodal data streams from multi-user VR environment that evaluate learners’ cognitive (clinical decision-making) and behavioral (situational awareness, communication) processes to provide data-informed feedback focused on improving team-based care of patients who suffer sudden medical emergencies. Through this new multimodal debriefing system, instructors will be able to provide personalized feedback to clinicians during post-simulation debriefing sessions. 

   more » « less
4. LogiDebrief: A Signal-Temporal Logic Based Automated Debriefing Approach with Large Language Models Integration

   Chen, Zirong; An, Ziyan; Reynolds, Jennifer; Mullen, Kristin; Martini, Stephen; Ma, Meiyi (August 2025, Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence)

   Emergency response services are critical to public safety, with 9-1-1 call-takers playing a key role in ensuring timely and effective emergency operations. To ensure call-taking performance consistency, quality assurance is implemented to evaluate and refine call-takers' skillsets. However, traditional human-led evaluations struggle with high call volumes, leading to low coverage and delayed assessments. We introduce LogiDebrief, an AI-driven framework that automates traditional 9-1-1 call debriefing by integrating Signal-Temporal Logic (STL) with Large Language Models (LLMs) for fully-covered rigorous performance evaluation. LogiDebrief formalizes call-taking requirements as logical specifications, enabling systematic assessment of 9-1-1 calls against procedural guidelines. It employs a three-step verification process: (1) contextual understanding to identify responder types, incident classifications, and critical conditions; (2) STL-based runtime checking with LLM integration to ensure compliance; and (3) automated aggregation of results into quality assurance reports. Beyond its technical contributions, LogiDebrief has demonstrated real-world impact. Successfully deployed at Metro Nashville Department of Emergency Communications, it has assisted in debriefing 1,701 real-world calls, saving 311.85 hours of active engagement. Empirical evaluation with real-world data confirms its accuracy, while a case study and extensive user study highlight its effectiveness in enhancing call-taking performance. 

   more » « less
5. Implementing Distributed Feedback in a Tool that Supports Peer-to-Peer Simulation in Healthcare

   Katz, Sandra; Albacete, Patricia; Jordan, Pamela; Silliman, Scott; Wrzesniewski, Matthew (June 2024, Springer, Cham)

   Sifaleras, A; Lin, F (Ed.)

   MedDbriefer is a web based ITS designed to enable healthcare students to do clinical scenarios anytime, anywhere. While one student “voice treats” a scenario’s patient(s) as the leader of a mock Emergency Medical Services (EMS) team, a peer records the team’s actions by using the system’s checklists, on a tablet. When the scenario ends, MedDbriefer analyzes the event log and generates a debriefing. MedDbriefer also provides a platform for research on simulation-based training. This paper describes how the system’s debriefing engine could be extended to deliver feedback during a scenario, as well as afterwards. MedDbriefer could then be used to compare the effectiveness of different ways of timing feedback delivery in computer-based simulation systems. 

   more » « less