More Like this

1. Collaborative Robotics Toolkit (CRTK): Open Software Framework for Surgical Robotics Research

   https://doi.org/10.1109/IRC.2020.00014  

   Su, Yun-Hsuan; Munawar, Adnan; Deguet, Anton; Lewis, Andrew; Lindgren, Kyle; Li, Yangming; Taylor, Russell H.; Fischer, Gregory S.; Hannaford, Blake; Kazanzides, Peter (November 2020, 2020 Fourth IEEE International Conference on Robotic Computing (IRC))

   null (Ed.)

   Robot-assisted minimally invasive surgery has made a substantial impact in operating rooms over the past few decades with their high dexterity, small tool size, and impact on adoption of minimally invasive techniques. In recent years, intelligence and different levels of surgical robot autonomy have emerged thanks to the medical robotics endeavors at numerous academic institutions and leading surgical robot companies. To accelerate interaction within the research community and prevent repeated development, we propose the Collaborative Robotics Toolkit (CRTK), a common API for the RAVEN-II and da Vinci Research Kit (dVRK) - two open surgical robot platforms installed at more than 40 institutions worldwide. CRTK has broadened to include other robots and devices, including simulated robotic systems and industrial robots. This common API is a community software infrastructure for research and education in cutting edge human-robot collaborative areas such as semi-autonomous teleoperation and medical robotics. This paper presents the concepts, design details and the integration of CRTK with physical robot systems and simulation platforms. 

   more » « less
2. From the Dexterous Surgical Skill to the Battlefield—A Robotics Exploratory Study

   https://doi.org/10.1093/milmed/usaa253  

   Gonzalez, Glebys T; Kaur, Upinder; Rahman, Masudur; Venkatesh, Vishnunandan; Sanchez, Natalia; Hager, Gregory; Xue, Yexiang; Voyles, Richard; Wachs, Juan (January 2021, Military Medicine)

   null (Ed.)

   ABSTRACT Introduction Short response time is critical for future military medical operations in austere settings or remote areas. Such effective patient care at the point of injury can greatly benefit from the integration of semi-autonomous robotic systems. To achieve autonomy, robots would require massive libraries of maneuvers collected with the goal of training machine learning algorithms. Although this is attainable in controlled settings, obtaining surgical data in austere settings can be difficult. Hence, in this article, we present the Dexterous Surgical Skill (DESK) database for knowledge transfer between robots. The peg transfer task was selected as it is one of the six main tasks of laparoscopic training. In addition, we provide a machine learning framework to evaluate novel transfer learning methodologies on this database. Methods A set of surgical gestures was collected for a peg transfer task, composed of seven atomic maneuvers referred to as surgemes. The collected Dexterous Surgical Skill dataset comprises a set of surgical robotic skills using the four robotic platforms: Taurus II, simulated Taurus II, YuMi, and the da Vinci Research Kit. Then, we explored two different learning scenarios: no-transfer and domain-transfer. In the no-transfer scenario, the training and testing data were obtained from the same domain; whereas in the domain-transfer scenario, the training data are a blend of simulated and real robot data, which are tested on a real robot. Results Using simulation data to train the learning algorithms enhances the performance on the real robot where limited or no real data are available. The transfer model showed an accuracy of 81% for the YuMi robot when the ratio of real-tosimulated data were 22% to 78%. For the Taurus II and the da Vinci, the model showed an accuracy of 97.5% and 93%, respectively, training only with simulation data. Conclusions The results indicate that simulation can be used to augment training data to enhance the performance of learned models in real scenarios. This shows potential for the future use of surgical data from the operating room in deployable surgical robots in remote areas. 

   more » « less
3. A Perspective on Miniature Soft Robotics: Actuation, Fabrication, Control, and Applications

   https://doi.org/10.1002/aisy.202300063  

   Chi, Yinding; Zhao, Yao; Hong, Yaoye; Li, Yanbin; Yin, Jie (April 2023, Advanced Intelligent Systems)

   Soft robotics enriches the robotic functionalities by engineering soft materials and electronics toward enhanced compliance, adaptivity, and friendly human machine. This decade has witnessed extraordinary progresses and benefits in scaling down soft robotics to small scale for a wide range of potential and promising applications, including medical and surgical soft robots, wearable and rehabilitation robots, and unconstructed environments exploration. This perspective highlights recent research efforts in miniature soft robotics in a brief and comprehensive way in terms of actuation, powering, designs, fabrication, control, and applications in four sections. Section 2 discusses the key aspects of materials selection and structural designs for small‐scale tethered and untethered actuation and powering, including fluidic actuation, stimuli‐responsive actuation, and soft living biohybrid materials, as well as structural forms from 1D to 3D. Section 3 discusses the advanced manufacturing techniques at small scales for fabricating miniature soft robots, including lithography, mechanical self‐assembly, additive manufacturing, tissue engineering, and other fabrication methods. Section 4 discusses the control systems used in miniature robots, including off‐board/onboard controls and artificial intelligence‐based controls. Section 5 discusses their potential broad applications in healthcare, small‐scale objects manipulating and processing, and environmental monitoring. Finally, outlooks on the challenges and opportunities are discussed. 

   more » « less
4. Real-Time Context-Aware Detection of Unsafe Events in Robot-Assisted Surgery

   https://doi.org/10.1109/DSN48063.2020.00054  

   Yasar, Mohammad Samin; Alemzadeh, Homa (June 2020, 2020 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN))

   Cyber-physical systems for robotic surgery have enabled minimally invasive procedures with increased precision and shorter hospitalization. However, with increasing complexity and connectivity of software and major involvement of human operators in the supervision of surgical robots, there remain significant challenges in ensuring patient safety. This paper presents a safety monitoring system that, given the knowledge of the surgical task being performed by the surgeon, can detect safety-critical events in real-time. Our approach integrates a surgical gesture classifier that infers the operational context from the time-series kinematics data of the robot with a library of erroneous gesture classifiers that given a surgical gesture can detect unsafe events. Our experiments using data from two surgical platforms show that the proposed system can detect unsafe events caused by accidental or malicious faults within an average reaction time window of 1,693 milliseconds and F1 score of 0.88 and human errors within an average reaction time window of 57 milliseconds and F1 score of 0.76. 

   more » « less
5. Expanding the Surgical Robotics Community: An Intuitive Sim-to-Real Control Framework for Raven-II with a Budget-Friendly Gamepad Controller

   https://doi.org/10.1109/ISMR63436.2024.10585789  

   Bui, Mai; Chalfant, Natalie; Sun, Cuiling; Fabrega, Sean; Peng, Haonan; Huang, Kevin; Su, Yun-Hsuan (June 2024, IEEE 2024 International Symposium on Medical Robotics (ISMR))

   This paper proposes a low-cost interface and refined digital twin for the Raven-II surgical robot. Previous simulations of the Raven-II, e.g. via the Asynchronous Multibody Framework (AMBF), presented salient drawbacks, including control inputs inconsistent with Raven-II software, and lack of stable, high-fidelity physical contact simulations. This work bridges both of these gaps, both (1) enabling robust, simulated contact mechanics for dynamic physical interactions with the Raven-II, and (2) developing a universal input format for both simulated and physical platforms. The method furthermore proposes a low cost, commodity game-controller interface for controlling both virtual and real realizations of Raven-II, thus greatly reducing the barrier to access for Raven-II research and collaboration. Overall, this work aims to eliminate the inconsistencies between simulated and real representations of the Raven-II. Such a development can expand the reach of surgical robotics research. Namely, providing end-to-end transparency between the simulated AMBF and physical Raven-II platforms enables a software testbed previously unavailable, e.g. for training real surgeons, for creating digital synthetic datasets, or for prototyping novel architectures like shared control strategies. Experiments validate this transparency by comparing joint trajectories between digital twin and physical testbed given identical inputs. This work may be extended and incorporated into recent efforts in developing modular or common software infrastructures for both simulation and control of real robotic devices, such as the Collaborative Robotics Toolkit (CRTK). 

   more » « less