ABSTRACT
Medical procedures are an essential part of healthcare delivery, and
the acquisition of procedural skills is a critical component of medical
education. Unfortunately, procedural skill is not evenly distributed
among medical providers. Skills may vary within departments
or institutions, and across geographic regions, depending on the
provider’s training and ongoing experience. We present a mixed
reality real-time communication system to increase access to procedural
skill training and to improve remote emergency assistance.
Our system allows a remote expert to guide a local operator through
a medical procedure. RGBD cameras capture a volumetric view of
the local scene including the patient, the operator, and the medical
equipment. The volumetric capture is augmented onto the remote
expert’s view to allow the expert to spatially guide the local operator
using visual and verbal instructions. We evaluated our mixed
reality communication system in a study in which experts teach the
ultrasound-guided placement of a central venous catheter (CVC) to
students in a simulation setting. The study compares state-of-theart
video communication against our system. The results indicate
that our system enhances and offers new possibilities for visual
communication compared to video teleconference-based training.
more »
« less
Volumetric Communication in Medical Training - CPR Emergencies
Over the last two years, we built, evaluated, and published a volumetric communication system for volumetric-based training and assistantship of medical personnel. Technical aspects of the system have been published at HICCS conference. In this paper, we discuss a follow-up work: the design and evaluation of a mixed reality real-time communication system for remote assistance during CPR emergencies. Our system allows an expert to guide a first responder, remotely, on how to give first aid. RGBD cameras capture a volumetric view of the local scene including the patient, the first responder, and the environment. The volumetric capture is augmented onto the remote expert's view to spatially guide the first responder using visual and verbal instructions. We evaluate the mixed reality communication system in a research study in which participants face a simulated emergency. The first responder moves the patient to the recovery position and performs chest compressions as well as mouth-to-mask ventilation. Our study compares mixed reality against videoconferencing-based assistance using CPR performance measures, cognitive workload surveys, and semi-structured interviews. We find that more visual communication including gestures and objects is used by the remote expert when assisting in mixed reality compared to videoconferencing. Moreover, the performance and the workload of the first responder during simulation do not differ significantly between the two technologies.
more »
« less
- NSF-PAR ID:
- 10469178
- Publisher / Repository:
- 15th International Conference on Education and New Learning Technologies
- Date Published:
- Page Range / eLocation ID:
- 7747 to 7747
- Format(s):
- Medium: X
- Location:
- Palma, Spain
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Work-in-Progress-Volumetric Communication for Remote Assistance Giving Cardiopulmonary ResuscitationWe present our work in progress, a real-time mixed reality communication system for remote assistance in medical emergency situations. 3D cameras capture the emergency situation and send volumetric data to a remote expert. The remote expert sees the volumetric scene through mixed reality glasses and guides an operator to the patient. The local operator receives audio and visual guidance augmented onto the mixed reality headset. We compare the mixed reality system against traditional video communication in a user study on a CPR emergency simulation. We evaluate task performance, cognitive load, and user interaction. The results will help to better understand the benefits of using augmented and volumetric information in medical emergency procedures.more » « less
-
We present a volumetric communication system that is designed for remote assistance of procedural tasks. The system allows a remote expert to visually guide a local operator. The two parties share a view that is spatially identical, but for the local operator it is of the object on which they operate, while for the remote expert, the object is presented as a mixed reality “hologram”. Guidance is provided by voice, gestures, and annotations performed directly on the object of interest or its hologram. At each end of the communication, spatial is visualized using mixed-reality glasses.more » « less
-
more » « less
Abstract Telementoring platforms can help transfer surgical expertise remotely. However, most telementoring platforms are not designed to assist in austere, pre-hospital settings. This paper evaluates the system for telementoring with augmented reality (STAR), a portable and self-contained telementoring platform based on an augmented reality head-mounted display (ARHMD). The system is designed to assist in austere scenarios: a stabilized first-person view of the operating field is sent to a remote expert, who creates surgical instructions that a local first responder wearing the ARHMD can visualize as three-dimensional models projected onto the patient’s body. Our hypothesis evaluated whether remote guidance with STAR could lead to performing a surgical procedure better, as opposed to remote audio-only guidance. Remote expert surgeons guided first responders through training cricothyroidotomies in a simulated austere scenario, and on-site surgeons evaluated the participants using standardized evaluation tools. The evaluation comprehended completion time and technique performance of specific cricothyroidotomy steps. The analyses were also performed considering the participants’ years of experience as first responders, and their experience performing cricothyroidotomies. A linear mixed model analysis showed that using STAR was associated with higher procedural and non-procedural scores, and overall better performance. Additionally, a binary logistic regression analysis showed that using STAR was associated to safer and more successful executions of cricothyroidotomies. This work demonstrates that remote mentors can use STAR to provide first responders with guidance and surgical knowledge, and represents a first step towards the adoption of ARHMDs to convey clinical expertise remotely in austere scenarios.
-
We design a volumetric communication system for remote assistance of procedural medical tasks. The system allows a remote expert to spatially guide a local operator using a real-time volumetric representation of the patient. Guidance is provided by voice, virtual hand metaphor, and annotations performed in situ. We include the feedback we received from the medical professionals and early NASA TLX [5] data on the cognitive load of the system.more » « less
