ABSTRACT
Medical procedures are an essential part of healthcare delivery, and
the acquisition of procedural skills is a critical component of medical
education. Unfortunately, procedural skill is not evenly distributed
among medical providers. Skills may vary within departments
or institutions, and across geographic regions, depending on the
provider’s training and ongoing experience. We present a mixed
reality real-time communication system to increase access to procedural
skill training and to improve remote emergency assistance.
Our system allows a remote expert to guide a local operator through
a medical procedure. RGBD cameras capture a volumetric view of
the local scene including the patient, the operator, and the medical
equipment. The volumetric capture is augmented onto the remote
expert’s view to allow the expert to spatially guide the local operator
using visual and verbal instructions. We evaluated our mixed
reality communication system in a study in which experts teach the
ultrasound-guided placement of a central venous catheter (CVC) to
students in a simulation setting. The study compares state-of-theart
video communication against our system. The results indicate
that our system enhances and offers new possibilities for visual
communication compared to video teleconference-based training.
more »
« less
Work-in-Progress-Volumetric Communication for Remote Assistance Giving Cardiopulmonary Resuscitation
We present our work in progress, a real-time mixed reality communication system for remote assistance in medical emergency situations. 3D cameras capture the emergency situation and send volumetric data to a remote expert. The remote expert sees the volumetric scene through mixed reality glasses and guides an operator to the patient. The local operator receives audio and visual guidance augmented onto the mixed reality headset. We compare the mixed reality system against traditional video communication in a user study on a CPR emergency simulation. We evaluate task performance, cognitive load, and user interaction. The results will help to better understand the benefits of using augmented and volumetric information in medical emergency procedures.
more »
« less
- Award ID(s):
- 2026505
- NSF-PAR ID:
- 10351678
- Date Published:
- Journal Name:
- 2022 8th International Conference of the Immersive Learning Research Network (iLRN)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Over the last two years, we built, evaluated, and published a volumetric communication system for volumetric-based training and assistantship of medical personnel. Technical aspects of the system have been published at HICCS conference. In this paper, we discuss a follow-up work: the design and evaluation of a mixed reality real-time communication system for remote assistance during CPR emergencies. Our system allows an expert to guide a first responder, remotely, on how to give first aid. RGBD cameras capture a volumetric view of the local scene including the patient, the first responder, and the environment. The volumetric capture is augmented onto the remote expert's view to spatially guide the first responder using visual and verbal instructions. We evaluate the mixed reality communication system in a research study in which participants face a simulated emergency. The first responder moves the patient to the recovery position and performs chest compressions as well as mouth-to-mask ventilation. Our study compares mixed reality against videoconferencing-based assistance using CPR performance measures, cognitive workload surveys, and semi-structured interviews. We find that more visual communication including gestures and objects is used by the remote expert when assisting in mixed reality compared to videoconferencing. Moreover, the performance and the workload of the first responder during simulation do not differ significantly between the two technologies.more » « less
-
We present a volumetric communication system that is designed for remote assistance of procedural tasks. The system allows a remote expert to visually guide a local operator. The two parties share a view that is spatially identical, but for the local operator it is of the object on which they operate, while for the remote expert, the object is presented as a mixed reality “hologram”. Guidance is provided by voice, gestures, and annotations performed directly on the object of interest or its hologram. At each end of the communication, spatial is visualized using mixed-reality glasses.more » « less
-
We design a volumetric communication system for remote assistance of procedural medical tasks. The system allows a remote expert to spatially guide a local operator using a real-time volumetric representation of the patient. Guidance is provided by voice, virtual hand metaphor, and annotations performed in situ. We include the feedback we received from the medical professionals and early NASA TLX [5] data on the cognitive load of the system.more » « less
-
We present the design of a mixed reality (MR) telehealth training system that aims to close the gap between in-person and distance training and re-training for medical procedures. Our system uses real-time volumetric capture as a means for communicating and relating spatial information between the non-colocated trainee and instructor. The system's design is based on a requirements elicitation study performed in situ, at a medical school simulation training center. The focus is on the lightweight real-time transmission of volumetric data - meaning the use of consumer hardware, easy and quick deployment, and low-demand computations. We evaluate the MR system design by analyzing the workload for the users during medical training. We compare in-person, video, and MR training workloads. The results indicate that the overall workload for central line placement training with MR does not increase significantly compared to video communication. Our work shows that, when designed strategically together with domain experts, an MR communication system can be used effectively for complex medical procedural training without increasing the overall workload for users significantly. Moreover, MR systems offer new opportunities for teaching due to spatial information, hand tracking, and augmented communication.more » « less