Generative Attention Learning (GenerAL) is a framework for high-DOF multi-fingered grasping that is not only robust to
dense clutter and novel objects but also effective with a variety of different parallel-jaw and multi-fingered robot hands. This
framework introduces a novel attention mechanism that substantially improves the grasp success rate in clutter. Its generative
nature allows the learning of full-DOF grasps with flexible end-effector positions and orientations, as well as all finger joint
angles of the hand. Trained purely in simulation, this framework skillfully closes the sim-to-real gap. To close the visual
sim-to-real gap, this framework uses a single depth image as input. To close the dynamics sim-to-real gap, this framework
circumvents continuous motor control with a direct mapping from pixel to Cartesian space inferred from the same depth
image. Finally, this framework demonstrates inter-robot generality by achieving over 92% real-world grasp success rates in
cluttered scenes with novel objects using two multi-fingered robotic hand-arm systems with different degrees of freedom.
more »
« less
Complex manipulation with a simple robotic hand through contact breaking and caging
Humans use all surfaces of the hand for contact-rich manipulation. Robot hands, in contrast, typically use only the fingertips, which can limit dexterity. In this work, we leveraged a potential energy–based whole-hand manipulation model, which does not depend on contact wrench modeling like traditional approaches, to design a robotic manipulator. Inspired by robotic caging grasps and the high levels of dexterity observed in human manipulation, a metric was developed and used in conjunction with the manipulation model to design a two-fingered dexterous hand, the Model W. This was accomplished by simulating all planar finger topologies composed of open kinematic chains of up to three serial revolute and prismatic joints, forming symmetric two-fingered hands, and evaluating their performance according to the metric. We present the best design, an unconventional robot hand capable of performing continuous object reorientation, as well as repeatedly alternating between power and pinch grasps—two contact-rich skills that have often eluded robotic hands—and we experimentally characterize the hand’s manipulation capability. This hand realizes manipulation motions reminiscent of thumb–index finger manipulative movement in humans, and its topology provides the foundation for a general-purpose dexterous robot hand.
more » « less- NSF-PAR ID:
- 10227816
- Publisher / Repository:
- American Association for the Advancement of Science (AAAS)
- Date Published:
- Journal Name:
- Science Robotics
- Volume:
- 6
- Issue:
- 54
- ISSN:
- 2470-9476
- Page Range / eLocation ID:
- Article No. eabd2666
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Traditional parallel-jaw grippers are insufficient for delicate object manipulation due to their stiffness and lack of dexterity. Other dexterous robotic hands often have bulky fingers, rely on complex time-varying cable drives, or are prohibitively expensive. In this paper, we introduce a novel low-cost compliant gripper with two centimeter-scaled 3-DOF delta robots using off-the-shelf linear actuators and 3D-printed soft materials. To model the kinematics of delta robots with soft compliant links, which diverge from typical rigid links, we train neural networks using a perception system. Furthermore, we analyze the delta robot’s force profile by varying the starting position in its workspace and measuring the resulting force from a push action. Finally, we demonstrate the compliance and dexterity of our gripper through six dexterous manipulation tasks involving small and delicate objects. Thus, we present the groundwork for creating modular multi-fingered hands that can execute precise and low-inertia manipulations.more » « less
-
To fully utilize the versatility of a multi-fingered dexterous robotic hand for executing diverse object grasps, one must consider the rich physical constraints introduced by hand-object interaction and object geometry. We propose an integrative approach of combining a generative model and a bilevel optimization (BO) to plan diverse grasp configurations on novel objects. First, a conditional variational autoencoder trained on merely six YCB objects predicts the finger placement directly from the object point cloud. The prediction is then used to seed a nonconvex BO that solves for a grasp configuration under collision, reachability, wrench closure, and friction constraints. Our method achieved an 86.7% success over 120 real world grasping trials on 20 household objects, including unseen and challenging geometries. Through quantitative empirical evaluations, we confirm that grasp configurations produced by our pipeline are indeed guaranteed to satisfy kinematic and dynamic constraints. A video summary of our results is available at youtu.be/9DTrImbN99I.more » « less
-
This paper explores the problem of autonomous, in-hand regrasping-the problem of moving from an initial grasp on an object to a desired grasp using the dexterity of a robot's fingers. We propose a planner for this problem which alternates between finger gaiting, and in-grasp manipulation. Finger gaiting enables the robot to move a single finger to a new contact location on the object, while the remaining fingers stably hold the object. In-grasp manipulation moves the object to a new pose relative to the robot's palm, while maintaining the contact locations between the hand and object. Given the object's geometry (as a mesh), the hand's kinematic structure, and the initial and desired grasps, we plan a sequence of finger gaits and object reposing actions to reach the desired grasp without dropping the object. We propose an optimization based approach and report in-hand regrasping plans for 5 objects over 5 in-hand regrasp goals each. The plans generated by our planner are collision free and guarantee kinematic feasibility.more » « less
-
null (Ed.)Achieving dexterous in-hand manipulation with robot hands is an extremely challenging problem, in part due to current limitations in hardware design. One notable bottleneck hampering the development of improved hardware for dexterous manipulation is the lack of a standardized benchmark for evaluating in-hand dexterity. In order to address this issue, we establish a new benchmark for evaluating in- hand dexterity, specifically for humanoid type robot hands: the Elliott and Connolly Benchmark. This benchmark is based on a classification of human manipulations established by Elliott and Connolly, and consists of 13 distinct in-hand manipulation patterns. We define qualitative and quantitative metrics for evaluation of the benchmark, and provide a detailed testing protocol. Additionally, we introduce a dexterous robot hand - the CMU Foam Hand III - which is evaluated using the benchmark, successfully completing 10 of the 13 manipulation patterns and outperforming human hand baseline results for several of the patterns.more » « less
