Robot teleoperation is an emerging field of study with wide applications in exploration, manufacturing, and healthcare, because it allows users to perform complex remote tasks while remaining distanced and safe. Haptic feedback offers an immersive user experience and expands the range of tasks that can be accomplished through teleoperation. In this paper, we present a novel wearable haptic feedback device for a teleoperation system that applies kinesthetic force feedback to the fingers of a user. The proposed device, called a ‘haptic muscle’, is a soft pneumatic actuator constructed from a fabric-silicone composite in a toroidal structure. We explore the requirements of the ideal haptic feedback mechanism, construct several haptic muscles using different materials, and experimentally determine their dynamic pressure response as well as sensitivity (their ability to communicate small changes in haptic feedback). Finally, we integrate the haptic muscles into a data glove and a teleoperation system and perform several user tests. Our results show that most users could detect detect force changes as low as 3% of the working range of the haptic muscles. We also find that the haptic feedback causes users to apply up to 52% less force on an object while handling soft and fragile objects with a teleoperation system.
more »
« less
A Haptic Interface for the Teleoperation of Extensible Continuum Manipulators
We describe a novel haptic interface designed specifically
for the teleoperation of extensible continuum manipulators.
The proposed device is based off of, and extends to the haptic
domain, a kinematically similar input device for continuum manipulators
called the MiniOct. This letter describes the physical design
of the new device, the method of creating impedance-type haptic
feedback to users, and some of the requirements for implementing
this device in a bilateral teleoperation scheme for continuum robots.
We report a series of initial experiments to validate the operation
of the system, including simulated and real-time conditions. The
experimental results show that a user can identify the direction
of planar obstacles from the feedback for both virtual and physical
environments. Finally, we discuss the challenges for providing
feedback to an operator about the state of a teleoperated continuum
manipulator.
more »
« less
- Award ID(s):
- 1718075
- NSF-PAR ID:
- 10169069
- Date Published:
- Journal Name:
- IEEE robotics automation letters
- Volume:
- 5
- Issue:
- 2
- ISSN:
- 2377-3766
- Page Range / eLocation ID:
- 1875-1882
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Abstract ROV operations are mainly performed via a traditional control kiosk and limited data feedback methods, such as the use of joysticks and camera view displays equipped on a surface vessel. This traditional setup requires significant personnel on board (POB) time and imposes high requirements for personnel training. This paper proposes a virtual reality (VR) based haptic-visual ROV teleoperation system that can substantially simplify ROV teleoperation and enhance the remote operator's situational awareness.
This study leverages the recent development in Mixed Reality (MR) technologies, sensory augmentation, sensing technologies, and closed-loop control, to visualize and render complex underwater environmental data in an intuitive and immersive way. The raw sensor data will be processed with physics engine systems and rendered as a high-fidelity digital twin model in game engines. Certain features will be visualized and displayed via the VR headset, whereas others will be manifested as haptic and tactile cues via our haptic feedback systems. We applied a simulation approach to test the developed system.
With our developed system, a high-fidelity subsea environment is reconstructed based on the sensor data collected from an ROV including the bathymetric, hydrodynamic, visual, and vehicle navigational measurements. Specifically, the vehicle is equipped with a navigation sensor system for real-time state estimation, an acoustic Doppler current profiler for far-field flow measurement, and a bio-inspired artificial literal-line hydrodynamic sensor system for near-field small-scale hydrodynamics. Optimized game engine rendering algorithms then visualize key environmental features as augmented user interface elements in a VR headset, such as color-coded vectors, to indicate the environmental impact on the performance and function of the ROV. In addition, augmenting environmental feedback such as hydrodynamic forces are translated into patterned haptic stimuli via a haptic suit for indicating drift-inducing flows in the near field. A pilot case study was performed to verify the feasibility and effectiveness of the system design in a series of simulated ROV operation tasks.
ROVs are widely used in subsea exploration and intervention tasks, playing a critical role in offshore inspection, installation, and maintenance activities. The innovative ROV teleoperation feedback and control system will lower the barrier for ROV pilot jobs.
-
null (Ed.)We present a novel haptic teleoperation approach that considers not only the safety but also the stability of a teleoperation system. Specifically, we build upon previous work on haptic shared control, which generates a reference haptic feedback that helps the human operator to safely navigate the robot but without taking away their control authority. Crucially, in this approach the force rendered to the user is not directly reflected in the motion of the robot (which is still directly controlled by the user); however, previous work in the area neglected to consider the possible instabilities in feedback loop generated by a user that over-responds to the haptic force. In this paper we introduce a differential constraint on the rendered force that makes the system finite-gain L2 stable; the constraint results in a Quadratically Constrained Quadratic Program (QCQP), for which we provide a closed-form solution. Our constraint is related to, but less restrictive than, the typical passivity constraint used in previous literature. We conducted an experimental simulation in which a human operator flies a UAV near an obstacle to evaluate the proposed method.more » « less
-
more » « less
Abstract In the realm of robotics and automation, robot teleoperation, which facilitates human–machine interaction in distant or hazardous settings, has surged in significance. A persistent issue in this domain is the delays between command issuance and action execution, causing negative repercussions on operator situational awareness, performance, and cognitive load. These delays, particularly in long-distance operations, are difficult to mitigate even with the most advanced computing advancements. Current solutions mainly revolve around machine-based adjustments to combat these delays. However, a notable lacuna remains in harnessing human perceptions for an enhanced subjective teleoperation experience. This paper introduces a novel approach of sensory manipulation for induced human adaptation in delayed teleoperation. Drawing from motor learning and rehabilitation principles, it is posited that strategic sensory manipulation, via altered sensory stimuli, can mitigate the subjective feeling of these delays. The focus is not on introducing new skills or adapting to novel conditions; rather, it leverages prior motor coordination experience in the context of delays. The objective is to reduce the need for extensive training or sophisticated automation designs. A human-centered experiment involving 41 participants was conducted to examine the effects of modified haptic cues in teleoperations with delays. These cues were generated from high-fidelity physics engines using parameters from robot-end sensors or physics engine simulations. The results underscored several benefits, notably the considerable reduction in task time and enhanced user perceptions about visual delays. Real-time haptic feedback, or the anchoring method, emerged as a significant contributor to these benefits, showcasing reduced cognitive load, bolstered self-confidence, and minimized frustration. Beyond the prevalent methods of automation design and training, this research underscores induced human adaptation as a pivotal avenue in robot teleoperation. It seeks to enhance teleoperation efficacy through rapid human adaptation, offering insights beyond just optimizing robotic systems for delay compensations.
-
Physical interaction between humans and robots can help robots learn to perform complex tasks. The robot arm gains information by observing how the human kinesthetically guides it throughout the task. While prior works focus on how the robot learns, it is equally important that this learning is transparent to the human teacher. Visual displays that show the robot’s uncertainty can potentially communicate this information; however, we hypothesize that visual feedback mechanisms miss out on the physical connection between the human and robot. In this work we present a soft haptic display that wraps around and conforms to the surface of a robot arm, adding a haptic signal at an existing point of contact without significantly affecting the interaction. We demonstrate how soft actuation creates a salient haptic signal while still allowing flexibility in device mounting. Using a psychophysics experiment, we show that users can accurately distinguish inflation levels of the wrapped display with an average Weber fraction of 11.4%. When we place the wrapped display around the arm of a robotic manipulator, users are able to interpret and leverage the haptic signal in sample robot learning tasks, improving identification of areas where the robot needs more training and enabling the user to provide better demonstrations. See videos of our device and user studies here: https://youtu.be/tX-2Tqeb9Nwmore » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1109/LRA.2020.2970642
