An official website of the United States government
Robot teleoperation is an emerging field of study with wide applications in exploration, manufacturing, and healthcare, because it allows users to perform complex remote tasks while remaining distanced and safe. Haptic feedback offers an immersive user experience and expands the range of tasks that can be accomplished through teleoperation. In this paper, we present a novel wearable haptic feedback device for a teleoperation system that applies kinesthetic force feedback to the fingers of a user. The proposed device, called a ‘haptic muscle’, is a soft pneumatic actuator constructed from a fabric-silicone composite in a toroidal structure. We explore the requirements of the ideal haptic feedback mechanism, construct several haptic muscles using different materials, and experimentally determine their dynamic pressure response as well as sensitivity (their ability to communicate small changes in haptic feedback). Finally, we integrate the haptic muscles into a data glove and a teleoperation system and perform several user tests. Our results show that most users could detect detect force changes as low as 3% of the working range of the haptic muscles. We also find that the haptic feedback causes users to apply up to 52% less force on an object while handling soft and fragile objects with a teleoperation system.
more »
« less
A Haptic Interface for the Teleoperation of Extensible Continuum Manipulators
We describe a novel haptic interface designed specifically for the teleoperation of extensible continuum manipulators. The proposed device is based off of, and extends to the haptic domain, a kinematically similar input device for continuum manipulators called the MiniOct. This letter describes the physical design of the new device, the method of creating impedance-type haptic feedback to users, and some of the requirements for implementing this device in a bilateral teleoperation scheme for continuum robots. We report a series of initial experiments to validate the operation of the system, including simulated and real-time conditions. The experimental results show that a user can identify the direction of planar obstacles from the feedback for both virtual and physical environments. Finally, we discuss the challenges for providing feedback to an operator about the state of a teleoperated continuum manipulator.
more »
« less
- Award ID(s):
- 1718075
- PAR ID:
- 10169069
- Date Published:
- Journal Name:
- IEEE robotics automation letters
- Volume:
- 5
- Issue:
- 2
- ISSN:
- 2377-3766
- Page Range / eLocation ID:
- 1875-1882
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
IntroductionAs robot teleoperation increasingly becomes integral in executing tasks in distant, hazardous, or inaccessible environments, operational delays remain a significant obstacle. These delays, inherent in signal transmission and processing, adversely affect operator performance, particularly in tasks requiring precision and timeliness. While current research has made strides in mitigating these delays through advanced control strategies and training methods, a crucial gap persists in understanding the neurofunctional impacts of these delays and the efficacy of countermeasures from a cognitive perspective. MethodsThis study addresses the gap by leveraging functional Near-Infrared Spectroscopy (fNIRS) to examine the neurofunctional implications of simulated haptic feedback on cognitive activity and motor coordination under delayed conditions. In a human-subject experiment (N= 41), sensory feedback was manipulated to observe its influences on various brain regions of interest (ROIs) during teleoperation tasks. The fNIRS data provided a detailed assessment of cerebral activity, particularly in ROIs implicated in time perception and the execution of precise movements. ResultsOur results reveal that the anchoring condition, which provided immediate simulated haptic feedback with a delayed visual cue, significantly optimized neural functions related to time perception and motor coordination. This condition also improved motor performance compared to the asynchronous condition, where visual and haptic feedback were misaligned. DiscussionThese findings provide empirical evidence about the neurofunctional basis of the enhanced motor performance with simulated synthetic force feedback in the presence of teleoperation delays. The study highlights the potential for immediate haptic feedback to mitigate the adverse effects of operational delays, thereby improving the efficacy of teleoperation in critical applications.more » « less
-
Microrobots powered by an external magnetic field could be used for sophisticated medical applications such as cell treatment, micromanipulation, and noninvasive surgery inside the body. Untethered microrobot applications can benefit from haptic technology and telecommunication, enabling telemedical micro-manipulation. Users can manipulate the microrobots with haptic feedback by interacting with the robot operating system remotely in such applications. Artificially created haptic forces based on wirelessly transmitted data and model-based guidance can aid human operators with haptic sensations while manipulating microrobots. The system presented here includes a haptic device and a magnetic tweezer system linked together using a network-based teleoperation method with motion models in fluids. The magnetic microrobots can be controlled remotely, and the haptic interactions with the remote environment can be felt in real time. A time-domain passivity controller is applied to overcome network delay and ensure stability of communication. This study develops and tests a motion model for microrobots and evaluates two image-based 3D tracking algorithms to improve tracking accuracy in various Newtonian fluids. Additionally, it demonstrates that microrobots can group together to transport multiple larger objects, move through microfluidic channels for detailed tasks, and use a novel method for disassembly, greatly expanding their range of use in microscale operations. Remote medical treatment in multiple locations, remote delivery of medication without the need for physical penetration of the skin, and remotely controlled cell manipulations are some of the possible uses of the proposed technology.more » « less
-
null (Ed.)We present a novel haptic teleoperation approach that considers not only the safety but also the stability of a teleoperation system. Specifically, we build upon previous work on haptic shared control, which generates a reference haptic feedback that helps the human operator to safely navigate the robot but without taking away their control authority. Crucially, in this approach the force rendered to the user is not directly reflected in the motion of the robot (which is still directly controlled by the user); however, previous work in the area neglected to consider the possible instabilities in feedback loop generated by a user that over-responds to the haptic force. In this paper we introduce a differential constraint on the rendered force that makes the system finite-gain L2 stable; the constraint results in a Quadratically Constrained Quadratic Program (QCQP), for which we provide a closed-form solution. Our constraint is related to, but less restrictive than, the typical passivity constraint used in previous literature. We conducted an experimental simulation in which a human operator flies a UAV near an obstacle to evaluate the proposed method.more » « less
-
Abstract ROV operations are mainly performed via a traditional control kiosk and limited data feedback methods, such as the use of joysticks and camera view displays equipped on a surface vessel. This traditional setup requires significant personnel on board (POB) time and imposes high requirements for personnel training. This paper proposes a virtual reality (VR) based haptic-visual ROV teleoperation system that can substantially simplify ROV teleoperation and enhance the remote operator's situational awareness. This study leverages the recent development in Mixed Reality (MR) technologies, sensory augmentation, sensing technologies, and closed-loop control, to visualize and render complex underwater environmental data in an intuitive and immersive way. The raw sensor data will be processed with physics engine systems and rendered as a high-fidelity digital twin model in game engines. Certain features will be visualized and displayed via the VR headset, whereas others will be manifested as haptic and tactile cues via our haptic feedback systems. We applied a simulation approach to test the developed system. With our developed system, a high-fidelity subsea environment is reconstructed based on the sensor data collected from an ROV including the bathymetric, hydrodynamic, visual, and vehicle navigational measurements. Specifically, the vehicle is equipped with a navigation sensor system for real-time state estimation, an acoustic Doppler current profiler for far-field flow measurement, and a bio-inspired artificial literal-line hydrodynamic sensor system for near-field small-scale hydrodynamics. Optimized game engine rendering algorithms then visualize key environmental features as augmented user interface elements in a VR headset, such as color-coded vectors, to indicate the environmental impact on the performance and function of the ROV. In addition, augmenting environmental feedback such as hydrodynamic forces are translated into patterned haptic stimuli via a haptic suit for indicating drift-inducing flows in the near field. A pilot case study was performed to verify the feasibility and effectiveness of the system design in a series of simulated ROV operation tasks. ROVs are widely used in subsea exploration and intervention tasks, playing a critical role in offshore inspection, installation, and maintenance activities. The innovative ROV teleoperation feedback and control system will lower the barrier for ROV pilot jobs.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1109/LRA.2020.2970642
