skip to main content

Title: Practical Approaches Towards Transparent and Stable Bilateral Teleoperation Under Time-Varying Network Delay

In real-life teleoperation scenarios, the presence of time-varying network delays, particularly in wireless networks, poses significant challenges in maintaining stability in a bilateral teleoperation system. Various approaches have been proposed in the past to address stability concerns; however, these often come at the expense of system transparency. Nevertheless, increasing transparency is crucial in a teleoperation system to enable precise and safe operations, as well as to provide real-time decision-making capabilities for the operator. This paper presents our comprehensive approaches to maximize teleoperation transparency by minimizing system impedance, enhance the wave variable method to handle time-varying network delays, and alleviate non-smooth effects caused by network jitters in bilateral teleoperation. The proposed methodologies take into account the real-world challenges and considerations to ensure the practical applicability and effectiveness of the teleoperation system. Throughout these implementations, passivity analysis is employed to ensure system stability, guaranteeing a reliable and safe teleoperation experience. The proposed approaches were successfully validated in Team Northeastern’s Avatar telepresence system, which achieved the 3rd place in ANA Avatar XPRIZE challenge.

more » « less
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
Springer Science + Business Media
Date Published:
Journal Name:
International Journal of Social Robotics
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. We describe a novel haptic interface designed specifically for the teleoperation of extensible continuum manipulators. The proposed device is based off of, and extends to the haptic domain, a kinematically similar input device for continuum manipulators called the MiniOct. This letter describes the physical design of the new device, the method of creating impedance-type haptic feedback to users, and some of the requirements for implementing this device in a bilateral teleoperation scheme for continuum robots. We report a series of initial experiments to validate the operation of the system, including simulated and real-time conditions. The experimental results show that a user can identify the direction of planar obstacles from the feedback for both virtual and physical environments. Finally, we discuss the challenges for providing feedback to an operator about the state of a teleoperated continuum manipulator. 
    more » « less
  2. Abstract

    Detection of deception attacks is pivotal to ensure the safe and reliable operation of cyber-physical systems (CPS). Detection of such attacks needs to consider time-series sequences and is very challenging especially for autonomous vehicles that rely on high-dimensional observations from camera sensors. The paper presents an approach to detect deception attacks in real-time utilizing sensor observations, with a special focus on high-dimensional observations. The approach is based on inductive conformal anomaly detection (ICAD) and utilizes a novel generative model which consists of a variational autoencoder (VAE) and a recurrent neural network (RNN) that is used to learn both spatial and temporal features of the normal dynamic behavior of the system. The model can be used to predict the observations for multiple time steps, and the predictions are then compared with actual observations to efficiently quantify the nonconformity of a sequence under attack relative to the expected normal behavior, thereby enabling real-time detection of attacks using high-dimensional sequential data. We evaluate the approach empirically using two simulation case studies of an advanced emergency braking system and an autonomous car racing example, as well as a real-world secure water treatment dataset. The experiments show that the proposed method outperforms other detection methods, and in most experiments, both false positive and false negative rates are less than 10%. Furthermore, execution times measured on both powerful cloud machines and embedded devices are relatively short, thereby enabling real-time detection.

    more » « less
  3. Soft robotics holds tremendous potential for various applications, especially in unstructured environments such as search and rescue operations. However, the lack of autonomy and teleoperability, limited capabilities, absence of gait diversity and real-time control, and onboard sensors to sense the surroundings are some of the common issues with soft-limbed robots. To overcome these limitations, we propose a spatially symmetric, topologically-stable, soft-limbed tetrahedral robot that can perform multiple locomotion gaits. We introduce a kinematic model, derive locomotion trajectories for different gaits, and design a teleoperation mechanism to enable real-time human-robot collaboration. We use the kinematic model to map teleoperation inputs and ensure smooth transitions between gaits. Additionally, we leverage the passive compliance and natural stability of the robot for toppling and obstacle navigation. Through experimental tests, we demonstrate the robot's ability to tackle various locomotion challenges, adapt to different situations, and navigate obstructed environments via teleoperation. 
    more » « less
  4. Abstract

    In the realm of robotics and automation, robot teleoperation, which facilitates human–machine interaction in distant or hazardous settings, has surged in significance. A persistent issue in this domain is the delays between command issuance and action execution, causing negative repercussions on operator situational awareness, performance, and cognitive load. These delays, particularly in long-distance operations, are difficult to mitigate even with the most advanced computing advancements. Current solutions mainly revolve around machine-based adjustments to combat these delays. However, a notable lacuna remains in harnessing human perceptions for an enhanced subjective teleoperation experience. This paper introduces a novel approach of sensory manipulation for induced human adaptation in delayed teleoperation. Drawing from motor learning and rehabilitation principles, it is posited that strategic sensory manipulation, via altered sensory stimuli, can mitigate the subjective feeling of these delays. The focus is not on introducing new skills or adapting to novel conditions; rather, it leverages prior motor coordination experience in the context of delays. The objective is to reduce the need for extensive training or sophisticated automation designs. A human-centered experiment involving 41 participants was conducted to examine the effects of modified haptic cues in teleoperations with delays. These cues were generated from high-fidelity physics engines using parameters from robot-end sensors or physics engine simulations. The results underscored several benefits, notably the considerable reduction in task time and enhanced user perceptions about visual delays. Real-time haptic feedback, or the anchoring method, emerged as a significant contributor to these benefits, showcasing reduced cognitive load, bolstered self-confidence, and minimized frustration. Beyond the prevalent methods of automation design and training, this research underscores induced human adaptation as a pivotal avenue in robot teleoperation. It seeks to enhance teleoperation efficacy through rapid human adaptation, offering insights beyond just optimizing robotic systems for delay compensations.

    more » « less
  5. Inertia from rotating masses of generators in power systems influence the instantaneous frequency change when an imbalance between electrical and mechanical power occurs. Renewable energy sources (RES), such as solar and wind power, are connected to the grid via electronic converters. RES connected through converters affect the system's inertia by decreasing it and making it time-varying. This new setting challenges the ability of current control schemes to maintain frequency stability. Proposing adequate controllers for this new paradigm is key for the performance and stability of future power grids. The contribution of this paper is a framework to learn sparse time-invariant frequency controllers in a power system network with a time-varying evolution of rotational inertia. We model power dynamics using a Switched-Affine hybrid system to consider different modes corresponding to different inertia coefficients. We design a controller that uses as features, i.e. input, the systems states. In other words, we design a control proportional to the angles and frequencies. We include virtual inertia in the controllers to ensure stability. One of our findings is that it is possible to restrict communication between the nodes by reducing the number of features in the controller (from 22 to 10 in our case study) without disrupting performance and stability. Furthermore, once communication between nodes has reached a threshold, increasing it beyond this threshold does not improve performance or stability. We find a correlation between optimal feature selection in sparse controllers and the topology of the network. 
    more » « less