Robots working in human environments often encounter a wide range of articulated objects, such as tools, cabinets, and other jointed objects. Such articulated objects can take an infinite number of possible poses, as a point in a potentially high-dimensional continuous space. A robot must perceive this continuous pose to manipulate the object to a desired pose. This problem of perception and manipulation of articulated objects remains a challenge due to its high dimensionality and multimodal uncertainty. Here, we describe a factored approach to estimate the poses of articulated objects using an efficient approach to nonparametric belief propagation. We consider inputs as geometrical models with articulation constraints and observed RGBD (red, green, blue, and depth) sensor data. The described framework produces object-part pose beliefs iteratively. The problem is formulated as a pairwise Markov random field (MRF), where each hidden node (continuous pose variable) is an observed object-part’s pose and the edges denote the articulation constraints between the parts. We describe articulated pose estimation by a “pull” message passing algorithm for nonparametric belief propagation (PMPNBP) and evaluate its convergence properties over scenes with articulated objects. Robot experiments are provided to demonstrate the necessity of maintaining beliefs to perform goal-driven manipulation tasks.
more »
« less
Characterizing Continuous Manipulation Families for Dexterous Soft Robot Hands
There has been an explosion of ideas in soft robotics over the past decade, resulting in unprecedented opportunities for end effector design. Soft robot hands offer benefits of low-cost, compliance, and customized design, with the promise of dexterity and robustness. The space of opportunities is vast and exciting. However, new tools are needed to understand the capabilities of such manipulators and to facilitate manipulation planning with soft manipulators that exhibit free-form deformations. To address this challenge, we introduce a sampling based approach to discover and model continuous families of manipulations for soft robot hands. We give an overview of the soft foam robots in production in our lab and describe novel algorithms developed to characterize manipulation families for such robots. Our approach consists of sampling a space of manipulation actions, constructing Gaussian Mixture Model representations covering successful regions, and refining the results to create continuous successful regions representing the manipulation family. The space of manipulation actions is very high dimensional; we consider models with and without dimensionality reduction and provide a rigorous approach to compare models across different dimensions by comparing coverage of an unbiased test dataset in the full dimensional parameter space. Results show that some dimensionality reduction is typically useful in populating the models, but without our technique, the amount of dimensionality reduction to use is difficult to predict ahead of time and can depend on the hand and task. The models we produce can be used to plan and carry out successful, robust manipulation actions and to compare competing robot hand designs.
more »
« less
- Award ID(s):
- 1925130
- NSF-PAR ID:
- 10293235
- Date Published:
- Journal Name:
- Frontiers in robotics and AI
- ISSN:
- 2296-9144
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)There has been great progress in soft robot design, manufacture, and control in recent years, and soft robots are a tool of choice for safe and robust handling of objects in conditions of uncertainty. Still, dexterous in-hand manipulation using soft robots remains a challenge. This paper introduces foam robot hands actuated by tendons sewn through a fabric glove. The flexibility of tendon actuation allows for high competence in utilizing deformation for robust in-hand manipulation. We discuss manufacturing, control, and design optimization for foam robots and demonstrate robust grasping and in-hand manipulation on a variety of different physical hand prototypes.more » « less
-
Robots working in human environments often encounter a wide range of articulated objects, such as tools, cabinets, and other jointed objects. Such articulated objects can take an infinite number of possible poses, as a point in a potentially high-dimensional continuous space. A robot must perceive this continuous pose in order to manipulate the object to a desired pose. This problem of perception and manipulation of articulated objects remains a challenge due to its high dimensionality and multi-modal uncertainty. In this paper, we propose a factored approach to estimate the poses of articulated objects using an efficient non-parametric belief propagation algorithm. We consider inputs as geometrical models with articulation constraints, and observed 3D sensor data. The proposed framework produces object-part pose beliefs iteratively. The problem is formulated as a pairwise Markov Random Field (MRF) where each hidden node (continuous pose variable) models an observed object-part's pose and each edge denotes an articulation constraint between a pair of parts. We propose articulated pose estimation by a Pull Message Passing algorithm for Nonparametric Belief Propagation (PMPNBP) and evaluate its convergence properties over scenes with articulated objects.more » « less
-
We build a system that enables any human to control a robot hand and arm, simply by demonstrating motions with their own hand. The robot observes the human operator via a single RGB camera and imitates their actions in real-time. Human hands and robot hands differ in shape, size, and joint structure, and performing this translation from a single uncalibrated camera is a highly underconstrained problem. Moreover, the retargeted trajectories must effectively execute tasks on a physical robot, which requires them to be temporally smooth and free of self-collisions. Our key insight is that while paired human-robot correspondence data is expensive to collect, the internet contains a massive corpus of rich and diverse human hand videos. We leverage this data to train a system that understands human hands and retargets a human video stream into a robot hand-arm trajectory that is smooth, swift, safe, and semantically similar to the guiding demonstration. We demonstrate that it enables previously untrained people to teleoperate a robot on various dexterous manipulation tasks. Our low-cost, glove-free, marker-free remote teleoperation system makes robot teaching more accessible and we hope that it can aid robots that learn to act autonomously in the real world.more » « less
-
Abstract Soft robots can undergo large elastic deformations and adapt to complex shapes. However, they lack the structural strength to withstand external loads due to the intrinsic compliance of fabrication materials (silicone or rubber). In this paper, we present a novel stiffness modulation approach that controls the robot’s stiffness on-demand without permanently affecting the intrinsic compliance of the elastomeric body. Inspired by concentric tube robots, this approach uses a Nitinol tube as the backbone, which can be slid in and out of the soft robot body to achieve robot pose or stiffness modulation. To validate the proposed idea, we fabricated a tendon-driven concentric tube (TDCT) soft robot and developed the model based on Cosserat rod theory. The model is validated in different scenarios by varying the joint-space tendon input and task-space external contact force. Experimental results indicate that the model is capable of estimating the shape of the TDCT soft robot with an average root-mean-square error (RMSE) of 0.90 (0.56% of total length) mm and average tip error of 1.49 (0.93% of total length) mm. Simulation studies demonstrate that the Nitinol backbone insertion can enhance the kinematic workspace and reduce the compliance of the TDCT soft robot by 57.7%. Two case studies (object manipulation and soft laparoscopic photodynamic therapy) are presented to demonstrate the potential application of the proposed design.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- The DOI is not currently available.
