More Like this

1. MedDbriefer: A Debriefing Research Platform and Tool to Support Peer-led Simulation-based Training in Healthcare

   Katz, S.; Albacete, P.; Jordan, P.; Silliman, S.; Yang, T. (January 2023, Proceedings of the Twenty-third International Conference of the Learning Sciences—ICLS 2023)

   Bilkstein, P.; Van Aaist, J.; Kizito, R.; Brennan, K. (Ed.)

   MedDbriefer allows paramedic students to engage in simulated prehospital emergency care scenarios and receive an automated debriefing on their performance. It is a web-based tool that runs on a tablet. Although debriefing is purported to be one of simulation-based training’s most critical components, there is little empirical research to guide human and automated debriefing. We implemented two approaches to debriefing in MedDbriefer and are conducting a randomized controlled trial to compare their effectiveness. 

   more » « less
2. Implementing Distributed Feedback in a Tool that Supports Peer-to-Peer Simulation in Healthcare

   Katz, Sandra; Albacete, Patricia; Jordan, Pamela; Silliman, Scott; Wrzesniewski, Matthew (June 2024, Springer, Cham)

   Sifaleras, A; Lin, F (Ed.)

   MedDbriefer is a web based ITS designed to enable healthcare students to do clinical scenarios anytime, anywhere. While one student “voice treats” a scenario’s patient(s) as the leader of a mock Emergency Medical Services (EMS) team, a peer records the team’s actions by using the system’s checklists, on a tablet. When the scenario ends, MedDbriefer analyzes the event log and generates a debriefing. MedDbriefer also provides a platform for research on simulation-based training. This paper describes how the system’s debriefing engine could be extended to deliver feedback during a scenario, as well as afterwards. MedDbriefer could then be used to compare the effectiveness of different ways of timing feedback delivery in computer-based simulation systems. 

   more » « less
3. LogiDebrief: A Signal-Temporal Logic Based Automated Debriefing Approach with Large Language Models Integration

   Chen, Zirong; An, Ziyan; Reynolds, Jennifer; Mullen, Kristin; Martini, Stephen; Ma, Meiyi (August 2025, Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence)

   Emergency response services are critical to public safety, with 9-1-1 call-takers playing a key role in ensuring timely and effective emergency operations. To ensure call-taking performance consistency, quality assurance is implemented to evaluate and refine call-takers' skillsets. However, traditional human-led evaluations struggle with high call volumes, leading to low coverage and delayed assessments. We introduce LogiDebrief, an AI-driven framework that automates traditional 9-1-1 call debriefing by integrating Signal-Temporal Logic (STL) with Large Language Models (LLMs) for fully-covered rigorous performance evaluation. LogiDebrief formalizes call-taking requirements as logical specifications, enabling systematic assessment of 9-1-1 calls against procedural guidelines. It employs a three-step verification process: (1) contextual understanding to identify responder types, incident classifications, and critical conditions; (2) STL-based runtime checking with LLM integration to ensure compliance; and (3) automated aggregation of results into quality assurance reports. Beyond its technical contributions, LogiDebrief has demonstrated real-world impact. Successfully deployed at Metro Nashville Department of Emergency Communications, it has assisted in debriefing 1,701 real-world calls, saving 311.85 hours of active engagement. Empirical evaluation with real-world data confirms its accuracy, while a case study and extensive user study highlight its effectiveness in enhancing call-taking performance. 

   more » « less
4. Debriefings on Prehospital Care Scenarios in MedDbriefer—A Tool to Support Peer Learning

   Katz, S.; Jordan, P.; Silliman, S. (January 2023, Proceedings of the 3rd International Conference on Novel and Intelligent Digital Systems, NIDS 2023)

   Across the healthcare professions, many students don’t get enough practice doing simulated clinical interactions during course labs to feel confident about passing certification exams and treating actual patients. To address this problem, we are developing MedDbriefer, a web-based tutoring system that runs on a tablet. MedDbriefer allows peers to engage in supplemental clinical scenarios on their own. With its current focus on paramedic train-ing, one student “voice treats” a simulated patient as the leader of a mock emergency medical services team while a peer uses MedDbriefer’s check-lists to log the team leader’s verbalized actions. The system then analyzes the event log and generates a debriefing, which highlights errors such as as-sessment actions and treatment interventions that the team leader missed or performed late. This paper focuses on how the system analyzes event logs to generate adaptive debriefings. 

   more » « less
5. Toward Wearable Devices for Multiteam Systems Learning

   Dubrow, S.; Bannan, B. (July 2019, Lecture notes in computer science)

   This chapter provides an overview of an exploratory case study involving a multiteam system in the fire and rescue emergency context incorporating human sensor analytics (e.g., proximity sensors) and other data sources to reveal important insights on within- and between-team learning and training. Incorporating a design research approach, the case study consisting of two live simulation scenarios that informed the design and development of a wearable technology-based system targeted to capture team-based behavior in the live simulation and visualize it during the debriefing session immediately following to potentially inform within- and cross-team behavior from a multiteam systems perspective informed by theory and practice. 

   more » « less