Search for: All records

Abstract Considering the criticality of post-simulation debriefings for skill development, more evidence is needed to establish how specific feedback design features might influence teams’ cognitive and metacognitive processing. The current research therefore investigates the effects of multisource feedback (MSF) and guided facilitation with video review, for both cognitive processing and reflective (meta-cognitive) behaviors during post-simulation debriefings. With a sample of 174 s-year dental students, randomly assigned to 20 teams, the authors conducted high-fidelity simulations of patient emergencies, followed by post-simulation debriefings, using a 2 × 2 factorial design to test the effects of MSF (present vs. absent) and guided facilitation with video review (present vs. absent). According to an ordered network analysis, designed to examine feedback processing levels (individual vs. team) and depth (high vs. low), as well as the presence of metacognitive reflective behaviors (evaluative behaviors, exploration of alternatives, decision-oriented behaviors), teams that received both MSF and guided facilitation demonstrated significantly deeper, team-level processing and more frequent evaluative behaviors. Teams that received only guided facilitation exhibited the highest rates of low-level, individual processing. However, facilitation also produced an additive effect that fostered reflection and a shift from individual- to team-oriented processing. In contrast, MSF alone produced the lowest levels of evaluative behaviors; without facilitation, it does not support team reflection. These results establish that combining MSF with guided facilitation and video review creates synergistic effects for team reflection. Even if MSF can highlight perceived performance discrepancies, teams need facilitation to interpret and learn collaboratively from the feedback. 

Background Team leadership during medical emergencies like cardiac arrest resuscitation is cognitively demanding, especially for trainees. These cognitive processes remain poorly characterized due to measurement challenges. Using virtual reality simulation, this study aimed to elucidate and compare communication and cognitive processes-such as decision-making, cognitive load, perceived pitfalls, and strategies-between expert and novice code team leaders to inform strategies for accelerating proficiency development. Methods A simulation-based mixed methods approach was utilized within a single large academic medical center, involving twelve standardized virtual reality cardiac arrest simulations. These 10- to 15-minutes simulation sessions were performed by seven experts and five novices. Following the simulations, a cognitive task analysis was conducted using a cued-recall protocol to identify the challenges, decision-making processes, and cognitive load experienced across the seven stages of each simulation. Results The analysis revealed 250 unique cognitive processes. In terms of reasoning patterns, experts used inductive reasoning, while novices tended to use deductive reasoning, considering treatments before assessments. Experts also demonstrated earlier consideration of potential reversible causes of cardiac arrest. Regarding team communication, experts reported more critical communications, with no shared subthemes between groups. Experts identified more teamwork pitfalls, and suggested more strategies compared to novices. For cognitive load, experts reported lower median cognitive load (53) compared to novices (80) across all stages, with the exception of the initial presentation phase. Conclusions The identified patterns of expert performance — superior teamwork skills, inductive clinical reasoning, and distributed cognitive strategiesn — can inform training programs aimed at accelerating expertise development. 

This study introduces AutoCLC, an AI-powered system designed to assess and provide feedback on closed-loop communication (CLC) in professional learning environments. CLC, where a sender’s Call-Out statement is acknowledged by the receiver’s Check-Back statement, is a critical safety protocol in high-reliability domains, including emergency medicine resuscitation teams. Existing methods for evaluating CLC lack quantifiable metrics and depend heavily on human observation. AutoCLC addresses these limitations by leveraging natural language processing and large language models to analyze audio recordings from Advanced Cardiovascular Life Support (ACLS) simulation training. The system identifies CLC instances, measures their frequency and rate per minute, and categorizes communications as effective, incomplete, or missed. Technical evaluations demonstrate AutoCLC achieves 78.9% precision for identifying Call-Outs and 74.3% for Check-Backs, with a performance gap of only 5% compared to human annotations. A user study involving 11 cardiac arrest instructors across three training sites supported the need for automated CLC assessment. Instructors found AutoCLC reports valuable for quantifying CLC frequency and quality, as well as for providing actionable, example-based feedback. Participants rated AutoCLC highly, with a System Usability Scale score of 76.4%, reflecting above-average usability. This work represents a significant step toward developing scalable, data-driven feedback systems that enhance individual skills and team performance in high-reliability settings. 

In clinical operations, teamwork can be the crucial factor that determines the final outcome. Prior studies have shown that sufficient collaboration is the key factor that determines the outcome of an operation. To understand how the team practices teamwork during the operation, we collected CliniDial from simulations of medical operations. CliniDial includes the audio data and its transcriptions, the simulated physiology signals of the patient manikins, and how the team operates from two camera angles. We annotate behavior codes following an existing framework to understand the teamwork process for CliniDial. We pinpoint three main characteristics of our dataset, including its label imbalances, rich and natural interactions, and multiple modalities, and conduct experiments to test existing LLMs’ capabilities on handling data with these characteristics. Experimental results show that CliniDial poses significant challenges to the existing models, inviting future effort on developing methods that can deal with real-world clinical data. We open-source the codebase at https: //github.com/MichiganNLP/CliniDial.† 

Simulation-based learning has become a cornerstone of healthcare education, fostering essential skills like communication, teamwork or decision-making in safe, controlled environments. However, participants’ reflection on simulations often rely on subjective recollections, limiting their effectiveness in promoting learning. This symposium explores how multimodal analytics and AI can enhance simulation-based education by automating teamwork analysis data, providing structured feedback, and supporting reflective practices. The papers examine real-time analytics for closed-loop communication in cardiac arrest simulations, multimodal data use to refine feedback in ICU nursing simulations, generative AI-powered chatbots facilitating nursing students' interpretation of multimodal learning analytics dashboards, and culturally sensitive, AI-based scenarios for Breaking Bad News in an Indian context. Collectively, these contributions highlight the transformative potential of using data and AI-enhanced solutions, emphasizing personalization, cultural sensitivity, and human-centered design, and invite dialogue on the pedagogical, technological and ethical implications of introducing data-based practices and AI-based tools in medical education. 

Clinical reasoning is a critical yet complex cognitive process of diagnostic and therapeutic decision-making in medical practice that has long challenged precise understanding and assessment. Sequential analysis can be used to uncover patterns and trends in clinical practices, contributing to improved training and ultimately leading to better patient care outcomes. In this study, 21 board-certified anesthesiologists participated in a simulated-based learning scenario requiring them to promptly recognize patient’s condition and initiate appropriate treatment. They were assigned into either the low-performing or high-performing group based on their performance. We utilized Markov Chain Transition Matrix, a robust statistical model for sequential data, to analyze participants’ practices using team reflection behavioral observation system and identified statistically significant differences between their transition matrices. The high-performing group had a much higher transition probability from evaluating information to implementation and from planning to planning. The implications are then discussed. 

Accounting neglect of intervention reasoning in CSCL research, we propose a new process model for team-based diagnostic and intervention reasoning in acute care, focusing on interactions among diagnostic activities (DAs), intervention activities (IAs), and collaborative activities (CAs) such as joint information processing, coordination, and communication. Using epistemic network analysis, we analyzed data from a VR-based cardiac arrest simulation to validate this model by comparing expert- and novice-led teams. As expected, expert-led teams demonstrated faster, more cohesive transitions between DAs and IAs, with a streamlined, linear CA pattern, while novice-led teams exhibited slower, fragmented transitions with cyclical CA patterns. These findings support the model’s potential to capture expertise-driven coordination and efficiency in high-stakes settings. Future research may expand this model across diverse team compositions and problem contexts. By refining understanding of acute care team dynamics, this model paves the way for instructional strategies enhancing coordination and performance in collaborative problem solving. 

This paper introduces iREACT, a novel VR simulation addressing key limitations in traditional cardiac arrest (CA) training. Conventional methods struggle to replicate the dynamic nature of real CA events, hindering Crew Resource Management (CRM) skill development. iREACT provides a non-linear, collaborative environment where teams respond to changing patient states, mirroring real CA complexities. By capturing multi-modal data (user actions, cognitive load, visual gaze) and offering real-time and post-session feedback, iREACT enhances CRM assessment beyond traditional methods. A formative evaluation with medical experts underscores its usability and educational value, with potential applications in other high-stakes training scenarios to improve teamwork, communication, and decision-making. 

Background Effective team communication is crucial for managing medical emergencies like malignant hyperthermia (MH), but current assessment methods fail to capture the dynamic and temporal nature of teamwork processes. The lack of reliable measures to inform feedback to teams is likely limiting the overall effectiveness of simulation training. This study demonstrates the application of ordered network analysis (ONA) to model communication sequences during the simulated MH scenario. Methods Twenty-two anesthesiologists participated in video-recorded MH simulations. Each scenario involved one participant as the primary anesthesiologist with confederates in supporting roles. Team communication was coded using the Team Reflection Behavioral Observation (TuRBO) framework, capturing behaviors related to information gathering, evaluation, planning, and implementation. ONA modeled the sequences of these coded behaviors as dynamic networks. Teams were classified as high- or low-performing based on timely dantrolene administration and appropriate MH treatment actions. Network visualizations and statistical tests compared communication patterns between groups. Results Five of 22 teams (23%) were high-performing. ONA revealed high-performers transitioned more effectively from situation assessment (information seeking/evaluation) to planning and implementation, while low-performers cycled between assessment behaviors without progressing (p = 0.04, Cohen’s d = 1.72). High-performers demonstrated stronger associations between invited input, explicitly assessing the situation, stating plans, and implementation. Conclusions Integrating video coding with ONA provides an innovative approach for examining team behaviors. Leveraging ONA can uncover patterns in communication timing and sequences, guiding targeted interventions to improve team coordination in various real-world clinical and simulated settings (e.g., operating room, EMS, ICU). 

Background: Over the past two decades, the use of Metaverse-enhanced simulations in medical education has witnessed significant advancement. These simulations offer immersive environments and technologies, such as augmented reality, virtual reality, and artificial intelligence that have the potential to revolutionize medical training by providing realistic, hands-on experiences in diagnosing and treating patients, practicing surgical procedures, and enhancing clinical decision-making skills. This scoping review aimed to examine the evolution of simulation technology and the emergence of metaverse applications in medical professionals' training, guided by Friedman's three dimensions in medical education: physical space, time, and content, along with an additional dimension of assessment. Methods: In this scoping review, we examined the related literature in six major databases including PubMed, EMBASE, CINAHL, Scopus, Web of Science, and ERIC. A total of 173 publications were selected for the final review and analysis. We thematically analyzed these studies by combining Friedman's three-dimensional framework with assessment. Results: Our scoping review showed that Metaverse technologies, such as virtual reality simulation and online learning modules have enabled medical education to extend beyond physical classrooms and clinical sites by facilitating remote training. In terms of the Time dimension, simulation technologies have made partial but meaningful progress in supplementing traditional time-dependent curricula, helping to shorten learning curves, and improve knowledge retention. As for the Content dimension, high-quality simulation and metaverse content require alignment with learning objectives, interactivity, and deliberate practice that should be developmentally integrated from basic to advanced skills. With respect to the Assessment dimension, learning analytics and automated metrics from metaverse-enabled simulation systems have enhanced competency evaluation and formative feedback mechanisms. However, their integration into high-stakes testing is limited, and qualitative feedback and human observation remain crucial. Conclusion: Our study provides an updated perspective on the achievements and limitations of using simulation to transform medical education, offering insights that can inform development priorities and research directions for human-centered, ethical metaverse applications that enhance healthcare professional training.