More Like this

1. Supervised Semi-Autonomous Control for Surgical Robot Based on Banoian Optimization

   https://doi.org/10.1109/IROS45743.2020.9341383  

   Chen, Junhong; Zhang, Dandan; Munawar, Adnan; Zhu, Ruiqi; Lo, Benny; Fischer, Gregory S.; Yang, Guang-Zhong (February 2021, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS))

   null (Ed.)

   The recent development of Robot-Assisted Minimally Invasive Surgery (RAMIS) has brought much benefit to ease the performance of complex Minimally Invasive Surgery (MIS) tasks and lead to more clinical outcomes. Compared to direct master-slave manipulation, semi-autonomous control for the surgical robot can enhance the efficiency of the operation, particularly for repetitive tasks. However, operating in a highly dynamic in-vivo environment is complex. Supervisory control functions should be included to ensure flexibility and safety during the autonomous control phase. This paper presents a haptic rendering interface to enable supervised semi-autonomous control for a surgical robot. Bayesian optimization is used to tune user-specific parameters during the surgical training process. User studies were conducted on a customized simulator for validation. Detailed comparisons are made between with and without the supervised semi-autonomous control mode in terms of the number of clutching events, task completion time, master robot end-effector trajectory and average control speed of the slave robot. The effectiveness of the Bayesian optimization is also evaluated, demonstrating that the optimized parameters can significantly improve users' performance. Results indicate that the proposed control method can reduce the operator's workload and enhance operation efficiency. 

   more » « less
2. Asynchronously Assigning, Monitoring, and Managing Assembly Goals in Virtual Reality for High-Level Robot Teleoperation

   https://doi.org/10.1109/VR58804.2024.00066  

   Aoyama, Shutaro; Liu, Jen-Shuo; Wang, Portia; Jain, Shreeya; Wang, Xuezhen; Xu, Jingxi; Song, Shuran; Tversky, Barbara; Feiner, Steven (March 2024, IEEE Virtual Reality 2024)

   We present a prototype virtual reality user interface for robot teleoperation that supports high-level specification of 3D object positions and orientations in remote assembly tasks. Users interact with virtual replicas of task objects. They asynchronously assign multiple goals in the form of 6DoF destination poses without needing to be familiar with specific robots and their capabilities, and manage and monitor the execution of these goals. The user interface employs two different spatiotemporal visualizations for assigned goals: one represents all goals within the user’s workspace (Aggregated View), while the other depicts each goal within a separate world in miniature (Timeline View). We conducted a user study of the interface without the robot system to compare how these visualizations affect user efficiency and task load. The results show that while the Aggregated View helped the participants finish the task faster, the participants preferred the Timeline View. 

   more » « less
3. iOS Based Pose Estimation and Gesture Recognition for Robot Manipulation

   https://doi.org/10.1109/SOSE55472.2022.9812685  

   Deshmukh, Nikhil; Savur, Celal; Sahin, Ferat (January 2022, 2022 17th Annual System of Systems Engineering Conference (SOSE))

   Interfacing between robots and humans has come a long way in the past few years, and new methods for smart, robust interaction are needed. Typically, a technician has to program a routine for a robot in order for the robot to be useful. This puts up a significant barrier to entry into the field of automating tasks using robots—not only is a technician and a computer required, but the robot is not adaptive to the immediate needs of the user. The robot is only capable of executing a pre-determined task and for any change to be made the entire system needs to be paused. This project seeks to bridge the gap between user and robot interface, creating an easy-to-use system that allows for adaptive robot control. Using a combination of computer vision and a monocular camera system and integrated LiDAR sensor on an iPhone, gesture recognition and pose estimation was conducted within an independent system to control the Baxter humanoid robot. The gathered data was sent wirelessly to the robot to be interpreted and then replay actions performed by the user. 

   more » « less
4. Continuous Gesture Control of a Robot Arm: Performance Is Robust to a Variety of Hand-to-Robot Maps

   https://doi.org/10.1109/TBME.2023.3323601  

   Khan, Steafan E; Danziger, Zachary C (March 2024, IEEE Transactions on Biomedical Engineering)

   Objective: Despite advances in human-machine interface design, we lack the ability to give people precise and fast control over high degree of freedom (DOF) systems, like robotic limbs. Attempts to improve control often focus on the static map that links user input to device commands; hypothesizing that the user’s skill acquisition can be improved by finding an intuitive map. Here we investigate what map features affect skill acquisition. Methods: Each of our 36 participants used one of three maps that translated their 19-dimensional finger movement into the 5 robot joints and used the robot to pick up and move objects. The maps were each constructed to maximize a different control principle to reveal what features are most critical for user performance. 1) Principal Components Analysis to maximize the linear capture of finger variance, 2) our novel Egalitarian Principal Components Analysis to maximize the equality of variance captured by each component and 3) a Nonlinear Autoencoder to achieve both high variance capture and less biased variance allocation across latent dimensions Results: Despite large differences in the mapping structures there were no significant differences in group performance. Conclusion: Participants’ natural aptitude had a far greater effect on performance than the map. Significance: Robot-user interfaces are becoming increasingly common and require new designs to make them easier to operate. Here we show that optimizing the map may not be the appropriate target to improve operator skill. Therefore, further efforts should focus on other aspects of the robot-user-interface such as feedback or learning environment. 

   more » « less
5. Ergodic Specifications for Flexible Swarm Control: From User Commands to Persistent Adaptation

   https://doi.org/10.15607/RSS.2020.XVI.067  

   Prabhakar, Ahalya; Abraham, Ian; Taylor, Annalisa; Schlafly, Millicent; Popovic, Katarina; Diniz, Giovani; Teich, Brendan; Simidchieva, Borislava; Clark, Shane; Murphey, Todd (July 2020, Robotics: Science and Systems)

   This paper presents a formulation for swarm control and high-level task planning that is dynamically responsive to user commands and adaptable to environmental changes. We design an end-to-end pipeline from a tactile tablet interface for user commands to onboard control of robotic agents based on decentralized ergodic coverage. Our approach demonstrates reliable and dynamic control of a swarm collective through the use of ergodic specifications for planning and executing agent trajectories as well as responding to user and external inputs. We validate our approach in a virtual reality simulation environment objectives in real-time. and in real-world experiments at the DARPA OFFSET Urban Swarm Challenge FX3 field tests with a robotic swarm where user-based control of the swarm and mission-based tasks require a dynamic and flexible response to changing conditions and objectives in real-time. 

   more » « less