An official website of the United States government
Abstract Successful surgical operations are characterized by preplanning routines to be executed during actual surgical operations. To achieve this, surgeons rely on the experience acquired from the use of cadavers, enabling technologies like virtual reality (VR) and clinical years of practice. However, cadavers, having no dynamism and realism as they lack blood, can exhibit limited tissue degradation and shrinkage, while current VR systems do not provide amplified haptic feedback. This can impact surgical training increasing the likelihood of medical errors. This work proposes a novel Mixed Reality Combination System (MRCS) that pairs Augmented Reality (AR) technology and an inertial measurement unit (IMU) sensor with 3D printed, collagen-based specimens that can enhance task performance like planning and execution. To achieve this, the MRCS charts out a path prior to a user task execution based on a visual, physical, and dynamic environment on the state of a target object by utilizing surgeon-created virtual imagery that, when projected onto a 3D printed biospecimen as AR, reacts visually to user input on its actual physical state. This allows a real-time user reaction of the MRCS by displaying new multi-sensory virtual states of an object prior to performing on the actual physical state of that same object enabling effective task planning. Tracked user actions using an integrated 9-Degree of Freedom IMU demonstrate task execution This demonstrates that a user, with limited knowledge of specific anatomy, can, under guidance, execute a preplanned task. In addition, to surgical planning, this system can be generally applied in areas such as construction, maintenance, and education.
more »
« less
Evaluation of Pre-Training with the da Vinci Skills Simulator on Motor Skill Acquisition in a Surgical Robotics Curriculum
Training for robotic surgery can be challenging due the complexity of the technology, as well as a high demand for the robotic systems that must be primarily used for clinical care. While robotic surgical skills are traditionally trained using the robotic hardware coupled with physical simulated tissue models and test-beds, there has been an increasing interest in using virtual reality simulators. Use of virtual reality (VR) comes with some advantages, such as the ability to record and track metrics associated with learning. However, evidence of skill transfer from virtual environments to physical robotic tasks has yet to be fully demonstrated. In this work, we evaluate the effect of virtual reality pre-training on performance during a standardized robotic dry-lab training curriculum, where trainees perform a set of tasks and are evaluated with a score based on completion time and errors made during the task. Results show that VR pre-training is weakly significant ([Formula: see text]) in reducing the number of repetitions required to achieve proficiency on the robotic task; however, it is not able to significantly improve performance in any robotic tasks. This suggests that important skills are learned during physical training with the surgical robotic system that cannot yet be replaced with VR training.
more »
« less
- PAR ID:
- 10377863
- Date Published:
- Journal Name:
- Journal of Medical Robotics Research
- Volume:
- 06
- Issue:
- 03n04
- ISSN:
- 2424-905X
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Purpose Teamwork in surgery depends on a shared mental model of success, i.e., a common understanding of objectives in the operating room. A shared model leads to increased engagement among team members and is associated with fewer complications and overall better outcomes for patients. However, clinical training typically focuses on role-specific skills, leaving individuals to acquire a shared model indirectly through on-the-job experience. Methods We investigate whether virtual reality (VR) cross-training, i.e, exposure to other roles, can enhance a shared mental model for non-surgeons more directly. Our study focuses on X-ray guided pelvic trauma surgery, a procedure where successful communication depends on the shared model between the surgeon and a C-arm technologist. We present a VR environment supporting both roles and evaluate a cross-training curriculum in which non-surgeons swap roles with the surgeon. Results Exposure to the surgical task resulted in higher engagement with the C-arm technologist role in VR, as measured by the mental demand and effort expended by participants. It also has a significant effect on non-surgeon’s mental model of the overall task; novice participants’ estimation of the mental demand and effort required for the surgeon’s task increases after training, while their perception of overall performance decreases, indicating a gap in understanding based solely on observation. This phenomenon was also present for a professional C-arm technologist. Conclusion Until now, VR applications for clinical training have focused on virtualizing existing curricula. We demonstrate how novel approaches which are not possible outside of a virtual environment, such as role swapping, may enhance the shared mental model of surgical teams by contextualizing each individual’s role within the overall task in a time- and cost-efficient manner. As workflows grow increasingly sophisticated, we see VR curricula as being able to directly foster a shared model for success, ultimately benefiting patient outcomes through more effective teamwork in surgery.more » « less
-
This paper introduces the Intelligent Learning Platform for Robotics Operations (IL-PRO), a Virtual Reality (VR) system designed to enhance robotics training in the Architecture, Engineering, and Construction (AEC) industry. IL-PRO addresses the growing need for effective training methods as the AEC sector adopts robotic automation. The system integrates VR technology with game-assisted learning, combining online multimedia lessons for theory with immersive VR tasks for practical skills. Developed iteratively using Design-Based Research principles, IL-PRO incorporates realistic robot simulations and progressive task complexity. The VR environment, built in Unity, aims to enhance engagement, motor coordination, and spatial awareness in robotics training. While future goals include AI-driven personalized instruction, this work-in-progress focuses on VR curriculum development and implementation. The paper concludes by discussing future directions, including curriculum expansion and cross-institutional adoption, to establish new benchmarks in innovative robotics education for the AEC industry.more » « less
-
Yan, C; Chai, H; Sun, T; Yuan, PF (Ed.)Abstract. The building industry is facing environmental, technological, and economic challenges, placing significant pressure on preparing the workforce for Industry 4.0 needs. The fields of Architecture, Engineering, and Construction (AEC) are being reshaped by robotics technologies which demand new skills and creating disruptive change to job markets. Addressing the learning needs of AEC students, professionals, and industry workers is critical to ensuring the competitiveness of the future workforce. In recent years advancements in Information Technology, Augmented Reality (AR), Virtual Reality (VR), and Artificial Intelligence (AI) have led to new research and theories on virtual learning environments. In the AEC fields researchers are beginning to rethink current robotics training to counteract costly and resource-intensive in-person learning. However, much of this work has been focused on simulation physics and has yet to adequately address how to engage AEC learners with different learning abilities, styles, and diverse backgrounds.This paper presents the advantages and difficulties associated with using new technologies to develop virtual reality (VR) learning games for robotics. It describes an ongoing project for creating performance driven curriculum. Drawing on the Constructivist Learning Theory, the affordances of Adaptive Learning Systems, and data collection methods from the VR game environment, the project provides a customized and performance-oriented approach to carrying out practical robotics tasks in real-world scenarios.more » « less
-
Impact of extended reality on robot-assisted surgery training: a systematic review and meta-analysisAbstract Robot-assisted surgeries (RAS) have an extremely steep learning curve. Because of this, surgeons have created many methods to practice RAS outside the operating room. These training models usually include animal or plastic models; however, extended reality simulators have recently been introduced into surgical training programs. This systematic review and meta-analysis was conducted to determine if extended reality simulators can improve the performance of robotic novices and how their performance compares to the conventional training of surgeons on surgical robots. Using the PRISMA 2020 guidelines, a systematic review was performed searching PubMed, Embase, Web of Science, and Cochrane library for studies that compared the performance of robotic novices that received no additional training, trained with extended reality, or trained with inanimate physical simulators (conventional additional training). Articles that gauged performance using GEARS or time to complete measurements were included, while articles that did not make this comparison were excluded. A meta-analysis was performed on the 15 studies found using SPSS to compare the performance outcomes of the novices after training. Robotic novices trained with extended reality simulators showed a statistically significant improvement in time to complete (Cohen’s d = −0.95,p = 0.02) compared to those with no additional training. Extended reality training also showed no statistically significant difference in performance in time to complete (Cohen’s d = 0.65,p = 0.14) or GEARS scores (Cohen’s d = −0.093, p = 0.34) compared to robotic novices trained with conventional models. This meta-analysis seeks to determine if extended reality simulators translate complex skills to surgeons in a low-cost and low-risk environment.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1142/S2424905X21500069
