skip to main content


Title: Contact Localization for Robot Arms in Motion without Torque Sensing
Detecting and localizing contacts is essential for robot manipulators to perform contact-rich tasks in unstructured environments. While robot skins can localize contacts on the surface of robot arms, these sensors are not yet robust or easily accessible. As such, prior works have explored using proprioceptive observations, such as joint velocities and torques, to perform contact localization. Many past approaches assume the robot is static during contact incident, a single contact is made at a time, or having access to accurate dynamics models and joint torque sensing. In this work, we relax these assumptions and propose using Domain Randomization to train a neural network to localize contacts of robot arms in motion without joint torque observations. Our method uses a novel cylindrical projection encoding of the robot arm surface, which allows the network to use convolution layers to process input features and transposed convolution layers to predict contacts. The trained network achieves a contact detection accuracy of 91.5% and a mean contact localization error of 3.0cm. We further demonstrate an application of the contact localization model in an obstacle mapping task, evaluated in both simulation and the real world.  more » « less
Award ID(s):
1956163
NSF-PAR ID:
10311420
Author(s) / Creator(s):
;
Date Published:
Journal Name:
IEEE International Conference on Robotics and Automation
ISSN:
1049-3492
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. —Robots often have to perform manipulation tasks in close proximity to people (Fig 1). As such, it is desirable to use a robot arm that has limited joint torques so as to not injure the nearby person. Unfortunately, these limited torques then limit the payload capability of the arm. By using contact with the environment, robots can expand their reachable workspace that, otherwise, would be inaccessible due to exceeding actuator torque limits. We adapt our recently developed INSAT algorithm [1] to tackle the problem of torque-limited whole arm manipulation planning through contact. INSAT requires no prior over contact mode sequence and no initial template or seed for trajectory optimization. INSAT achieves this by interleaving graph search to explore the manipulator joint configuration space with incre- mental trajectory optimizations seeded by neighborhood solutions to find a dynamically feasible trajectory through contact. We demonstrate our results on a variety of manipulators and scenarios in simulation. We also experimentally show our planner exploiting robot-environment contact for the pick and place of a payload using a Kinova Gen3 robot. In comparison to the same trajectory running in free space, we experimentally show that the utilization of bracing contacts reduces the overall torque required to execute the trajectory. 
    more » « less
  2. Tan, Jie ; Toussaint, Marc ; Darvish, Kourosh (Ed.)
    Contacts play a critical role in most manipulation tasks. Robots today mainly use proximal touch/force sensors to sense contacts, but the information they provide must be calibrated and is inherently local, with practical applications relying either on extensive surface coverage or restrictive assumptions to resolve ambiguities. We propose a vision-based extrinsic contact localization task: with only a single RGB-D camera view of a robot workspace, identify when and where an object held by the robot contacts the rest of the environment. We show that careful task-attuned design is critical for a neural network trained in simulation to discover solutions that transfer well to a real robot. Our final approach im2contact demonstrates the promise of versatile general-purpose contact perception from vision alone, performing well for localizing various contact types (point, line, or planar; sticking, sliding, or rolling; single or multiple), and even under occlusions in its camera view. Video results can be found at: https://sites.google.com/view/im2contact/home 
    more » « less
  3. Rehabilitation is a crucial process for patients suffering from motor disorders. The current practice is performing rehabilitation exercises under clinical expert supervision. New approaches are needed to allow patients to perform prescribed exercises at their homes and alleviate commuting requirements, expert shortages, and healthcare costs. Human joint estimation is a substantial component of these programs since it offers valuable visualization and feedback based on body movements. Camera-based systems have been popular for capturing joint motion. However, they have high-cost, raise serious privacy concerns, and require strict lighting and placement settings. We propose a millimeter-wave (mmWave)-based assistive rehabilitation system (MARS) for motor disorders to address these challenges. MARS provides a low-cost solution with a competitive object localization and detection accuracy. It first maps the 5D time-series point cloud from mmWave to a lower dimension. Then, it uses a convolution neural network (CNN) to estimate the accurate location of human joints. MARS can reconstruct 19 human joints and their skeleton from the point cloud generated by mmWave radar. We evaluate MARS using ten specific rehabilitation movements performed by four human subjects involving all body parts and obtain an average mean absolute error of 5.87 cm for all joint positions. To the best of our knowledge, this is the first rehabilitation movements dataset using mmWave point cloud. MARS is evaluated on the Nvidia Jetson Xavier-NX board. Model inference takes only 64 s and consumes 442 J energy. These results demonstrate the practicality of MARS on low-power edge devices. 
    more » « less
  4. Abstract

    We present experimental evidence for a new mechanism for how smooth surfaces emerge during repetitive sliding contacts, as in polishing. Electron microscopy observations of Ti-6Al-4V surface with a spherical asperity structure—realized via additive manufacturing—during successive polishing stages suggest that asperity-abrasive contacts exhibit viscous behavior, where the asperity material flows in the form of thin (1–10 μm) fluid-like layers. Subsequent bridging of these layers among neighboring asperities results in progressive surface smoothening. Using analytical asperity-abrasive contact temperature modeling and microstructural characterization, we show that the sliding contacts encounter flash temperatures of the order of 700–900 K which is in the range of the dynamic recrystallization temperature of the material considered, thus supporting the experimental observations. Besides providing a new perspective on the long-held mechanism of polishing, our observations provide a novel approach based on graph theory to quantitatively characterize the evolution of surface morphology. Results suggest that the graph representation offers a more efficient measure to characterize the surface morphology emerging at various stages of polishing. The research findings and observations are of broad relevance to the understanding of plastic flow behavior of sliding contacts ubiquitous in materials processing, tribology, and natural geological processes as well as present unique opportunities to tailor the microstructures by controlling the thermomechanics of the asperity-abrasive contacts.

     
    more » « less
  5. This paper proposes a new method to measure the pose and localize the contacts with the surrounding environment for an inflatable soft robot by using optical sensors (photocells), inertial measurement units (IMUs), and a pressure sensor. These affordable sensors reside entirely aboard the robot and will be effective in environments where external sensors, such as motion capture, are not feasible to use. The entire bore of the robot is used as a waveguide to transfer the light. When the robot is working, the photocell signals vary with the current shape of the robot and the IMUs measure the orientation of its tip. Analytical functions are developed to relate the photocell signals and the robot pose. Since the soft robot is deformable, the occurrence of contact at any location on its body will modify the sensor signals. This simple measurement approach generates enough information to allow contact events to be detected and classified with high precision using a machine learning algorithm. 
    more » « less