Cyber-physical systems for robotic surgery have enabled minimally invasive procedures with increased precision and shorter hospitalization. However, with increasing complexity and connectivity of software and major involvement of human operators in the supervision of surgical robots, there remain significant challenges in ensuring patient safety. This paper presents a safety monitoring system that, given the knowledge of the surgical task being performed by the surgeon, can detect safety-critical events in real-time. Our approach integrates a surgical gesture classifier that infers the operational context from the time-series kinematics data of the robot with a library of erroneous gesture classifiers that given a surgical gesture can detect unsafe events. Our experiments using data from two surgical platforms show that the proposed system can detect unsafe events caused by accidental or malicious faults within an average reaction time window of 1,693 milliseconds and F1 score of 0.88 and human errors within an average reaction time window of 57 milliseconds and F1 score of 0.76.
more »
« less
Design And Development of a Novel Assistive Device for Laparoscopic Surgery Using Granular Jamming
Laparoscopic surgery has a notably high learning curve, hindering typical approaches to training. Due to unique challenges that are not present in open surgery (the hinge effect, small field of view (FoV), lack of depth perception, and small workspace), a surgical resident may be delayed in participating in laparoscopic surgery until later in residency. Having a narrow window to complete highly specialized training can lead to graduates feeling under-prepared for solo practice. Additionally, delayed introduction may expose trainees to fewer than 200 laparoscopic cases. Therefore, there is a need for surgical residents to increase both their caseload and training window without compromising patient safety. This project aims to develop and test a proof-of-concept prototype that uses granular jamming technology to controllably vary the force required to move a laparoscopic tool. By increasing tool resistance, the device helps prevents accidental injury to important nearby anatomical structures such as urinary tract, vasculature, and/or bowel. Increasing the safety of laparoscopic surgery would allow residents to begin their training earlier, gaining exposure and confidence. A device to adjust tool resistance has benefits to the experienced surgeon as well – surgeries require continuous tool adjustment and tension, resulting in fatigue. Increasing tool resistance can assist surgeons in situations requiring continuous tension and can also provide safety against sudden movements. This investigational device was prototyped using SolidWorks CAD software, then 3D printed and assessed with a laparoscopic box trainer.
more » « less- Award ID(s):
- 2026516
- PAR ID:
- 10398119
- Date Published:
- Journal Name:
- 2022 Design of Medical Devices Conference
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Laparoscopic surgery presents practical benefits over traditional open surgery, including reduced risk of infection, discomfort and recovery time for patients. Introducing robot systems into surgical tasks provides additional enhancements, including improved precision, remote operation, and an intelligent software layer capable of filtering aberrant motion and scaling surgical maneuvers. However, the software interface in telesurgery also lends itself to potential adversarial cyber attacks. Such attacks can negatively effect both surgeon motion commands and sensory information relayed to the operator. To combat cyber attacks on the latter, one method to enhance surgeon feedback through multiple sensory pathways is to incorporate reliable, complementary forms of information across different sensory modes. Built-in partial redundancies or inferences between perceptual channels, or perception complementarities, can be used both to detect and recover from compromised operator feedback. In surgery, haptic sensations are extremely useful for surgeons to prevent undue and unwanted tissue damage from excessive tool-tissue force. Direct force sensing is not yet deployable due to sterilization requirements of the operating room. Instead, combinations of other sensing methods may be relied upon, such as noncontact model-based force estimation. This paper presents the design of a surgical simulator software that can be used for vision-based non-contact force sensing to inform the perception complementarity of vision and force feedback for telesurgery. A brief user study is conducted to verify the efficacy of graphical force feedback from vision-based force estimation, and suggests that vision may effectively complement direct force sensing.more » « less
-
Trocar insertion is a critical first step of all minimally invasive surgery; however, it also carries a high risk for errors. Studies suggest that entry errors are the most common complication in laparoscopic surgery with 4% of errors leading to patient fatality. Surgeon error due to excessive force is often the cause for entry errors; however, adequate training has been shown to reduce the risk of these surgical errors. In practice, institutions lack widespread and relatively inexpensive means to train surgeons for trocar entry that does not involve patient risk. In our prior work, we presented a simple Stewart platform haptic device with a numerical model to simulate key force characteristics of trocar insertion. Evaluation in our first study was limited to device characterization. In this paper, we present a more robust haptic mechanism with higher fidelity linear actuators, an increased workspace, and tissue visualization to accompany haptic cues. We also present a novel upper module that allows for a sudden drop of the trocar after the final puncture event to create a more realistic simulation. We performed a user study with eight novices to investigate how well the device and visualization train users in the trocar insertion procedure. By the end of the experiment, subjects using the device had a normalized error reduction of roughly 85% on average, relative to themselves. This device shows potential for widespread training of trocar insertion, possibly leading to fewer complications and deaths following the procedure. Finally, our upper module also represents an innovative addition for traditional admittance-type haptic device designs, not typically capable of accurately representing motion in free space.more » « less
-
While robot-assisted minimally invasive surgery (RMIS) procedures afford a variety of benefits over open surgery and manual laparoscopic operations (including increased tool dexterity, reduced patient pain, incision size, trauma and recovery time, and lower infection rates [ 1 ], lack of spatial awareness remains an issue. Typical laparoscopic imaging can lack sufficient depth cues and haptic feedback, if provided, rarely reflects realistic tissue–tool interactions. This work is part of a larger ongoing research effort to reconstruct 3D surfaces using multiple viewpoints in RMIS to increase visual perception. The manual placement and adjustment of multicamera systems in RMIS are nonideal and prone to error [ 2 ], and other autonomous approaches focus on tool tracking and do not consider reconstruction of the surgical scene [ 3 , 4 , 5 ]. The group’s previous work investigated a novel, context-aware autonomous camera positioning method [ 6 ], which incorporated both tool location and scene coverage for multiple camera viewpoint adjustments. In this paper, the authors expand upon this prior work by implementing a streamlined deep reinforcement learning approach between optimal viewpoints calculated using the prior method [ 6 ] which encourages discovery of otherwise unobserved and additional camera viewpoints. Combining the framework and robustness of the previous work with the efficiency and additional viewpoints of the augmentations presented here results in improved performance and scene coverage promising towards real-time implementation.more » « less
-
null (Ed.)The unprecedented shock caused by the COVID-19 pandemic has severely influenced the delivery of regular healthcare services. Most non-urgent medical activities, including elective surgeries, have been paused to mitigate the risk of infection and to dedicate medical resources to managing the pandemic. In this regard, not only surgeries are substantially influenced, but also pre- and post-operative assessment of patients and training for surgical procedures have been significantly impacted due to the pandemic. Many countries are planning a phased reopening, which includes the resumption of some surgical procedures. However, it is not clear how the reopening safe-practice guidelines will impact the quality of healthcare delivery. This perspective article evaluates the use of robotics and AI in 1) robotics-assisted surgery, 2) tele-examination of patients for pre- and post-surgery, and 3) tele-training for surgical procedures. Surgeons interact with a large number of staff and patients on a daily basis. Thus, the risk of infection transmission between them raises concerns. In addition, pre- and post-operative assessment also raises concerns about increasing the risk of disease transmission, in particular, since many patients may have other underlying conditions, which can increase their chances of mortality due to the virus. The pandemic has also limited the time and access that trainee surgeons have for training in the OR and/or in the presence of an expert. In this article, we describe existing challenges and possible solutions and suggest future research directions that may be relevant for robotics and AI in addressing the three tasks mentioned above.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1115/DMD2022-1007
