More Like this

1. Collaborative System Design of Mixed Reality Communication for Medical Training

   Rebol, Manuel Pietroszek (January 2023, Proceedings of the Annual Hawaii International Conference on System Sciences)

   We present the design of a mixed reality (MR) telehealth training system that aims to close the gap between in-person and distance training and re-training for medical procedures. Our system uses real-time volumetric capture as a means for communicating and relating spatial information between the non-colocated trainee and instructor. The system's design is based on a requirements elicitation study performed in situ, at a medical school simulation training center. The focus is on the lightweight real-time transmission of volumetric data - meaning the use of consumer hardware, easy and quick deployment, and low-demand computations. We evaluate the MR system design by analyzing the workload for the users during medical training. We compare in-person, video, and MR training workloads. The results indicate that the overall workload for central line placement training with MR does not increase significantly compared to video communication. Our work shows that, when designed strategically together with domain experts, an MR communication system can be used effectively for complex medical procedural training without increasing the overall workload for users significantly. Moreover, MR systems offer new opportunities for teaching due to spatial information, hand tracking, and augmented communication. 

   more » « less
2. Evaluating Augmented Reality Communication: How Can We Teach Procedural Skill in AR?

   Rebol, M; Pietroszek, D; Sikka, N; Ranniger, C; Hood, C; Rutenberg, A; Sasankan, P; Gütl, C (October 2023, VRST '23: Proceedings of the 29th ACM Symposium on Virtual Reality Software and Technology)

   Augmented reality (AR) has great potential for use in healthcare applications, especially remote medical training and supervision. In this paper, we analyze the usage of an AR communication system to teach a medical procedure, the placement of a central venous catheter (CVC) under ultrasound guidance. We examine various AR communication and collaboration components, including gestural communication, volumetric information, annotations, augmented objects, and augmented screens. We compare how teaching in AR differs from teaching through videoconferencing-based communication. Our results include a detailed medical training steps analysis in which we compare how verbal and visual communication differs between video and AR training. We identify procedural steps in which medical experts give visual instructions utilizing AR components. We examine the change in AR usage and interaction over time and recognize patterns between users. Moreover, AR design recommendations are given based on post-training interviews. 

   more » « less
3. Volumetric Communication in Medical Training - CPR Emergencies

   https://doi.org/10.21125/edulearn.2023.2014  

   Pietroszek, Krzysztof; Rebol, Manuel; Steinmaurer, Alexander; Gamillscheg, Florian; Gütl, Christian; Ranniger, Claudia; Hood, Colton; Rutenberg, Adam; Sikka, Neal (July 2023, 15th International Conference on Education and New Learning Technologies)

   Over the last two years, we built, evaluated, and published a volumetric communication system for volumetric-based training and assistantship of medical personnel. Technical aspects of the system have been published at HICCS conference. In this paper, we discuss a follow-up work: the design and evaluation of a mixed reality real-time communication system for remote assistance during CPR emergencies. Our system allows an expert to guide a first responder, remotely, on how to give first aid. RGBD cameras capture a volumetric view of the local scene including the patient, the first responder, and the environment. The volumetric capture is augmented onto the remote expert's view to spatially guide the first responder using visual and verbal instructions. We evaluate the mixed reality communication system in a research study in which participants face a simulated emergency. The first responder moves the patient to the recovery position and performs chest compressions as well as mouth-to-mask ventilation. Our study compares mixed reality against videoconferencing-based assistance using CPR performance measures, cognitive workload surveys, and semi-structured interviews. We find that more visual communication including gestures and objects is used by the remote expert when assisting in mixed reality compared to videoconferencing. Moreover, the performance and the workload of the first responder during simulation do not differ significantly between the two technologies. 

   more » « less
4. Mixed Reality Communication for Medical Procedures: Teaching the Placement of a Central Venous Catheter

   https://doi.org/10.1109/ISMAR55827.2022.00050  

   Rebol, Manuel; Pietroszek, Krzysztof; Ranniger, Claudia; Hood, Colton; Rutenberg, Adam; Sikka, Neal; Li, David; Gütl, Christian (October 2022, 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR))

   ABSTRACT Medical procedures are an essential part of healthcare delivery, and the acquisition of procedural skills is a critical component of medical education. Unfortunately, procedural skill is not evenly distributed among medical providers. Skills may vary within departments or institutions, and across geographic regions, depending on the provider’s training and ongoing experience. We present a mixed reality real-time communication system to increase access to procedural skill training and to improve remote emergency assistance. Our system allows a remote expert to guide a local operator through a medical procedure. RGBD cameras capture a volumetric view of the local scene including the patient, the operator, and the medical equipment. The volumetric capture is augmented onto the remote expert’s view to allow the expert to spatially guide the local operator using visual and verbal instructions. We evaluated our mixed reality communication system in a study in which experts teach the ultrasound-guided placement of a central venous catheter (CVC) to students in a simulation setting. The study compares state-of-theart video communication against our system. The results indicate that our system enhances and offers new possibilities for visual communication compared to video teleconference-based training. 

   more » « less
5. Work-in-Progress-Volumetric Communication for Remote Assistance Giving Cardiopulmonary Resuscitation

   https://doi.org/10.23919/iLRN55037.2022.9815989  

   Rebol, M; Pietroszek, K; Ranniger, C; Hood, V; Rutenberg, A; Sikka, N; Steinmaurer, A; Gutl, C (May 2022, IEEE)

   We present our work in progress, a real-time mixed reality communication system for remote assistance in medical emergency situations. 3D cameras capture the emergency situa-tion and send volumetric data to a remote expert. The remote expert sees the volumetric scene through mixed reality glasses and guides an operator at the patient. The local operator receives audio and visual guidance augmented onto the mixed reality headset. We compare the mixed reality system against traditional video communication in a user study on a CPR emergency simulation. We evaluate task performance, cognitive load, and user interaction. The results will help to better understand the benefits of using augmented and volumetric information in medical emergency procedures. 

   more » « less