An official website of the United States government
Task-relevant grasping is critical for industrial assembly, where downstream manipulation tasks constrain the set of valid grasps. Learning how to perform this task, however, is challenging, since task-relevant grasp labels are hard to define and annotate. There is also yet no consensus on proper representations for modeling or off-the-shelf tools for performing task-relevant grasps. This work proposes a framework to learn task-relevant grasping for industrial objects without the need of time-consuming real-world data collection or manual annotation. To achieve this, the entire framework is trained solely in simulation, including supervised training with synthetic label generation and self-supervised, hand-object interaction. In the context of this framework, this paper proposes a novel, object-centric canonical representation at the category level, which allows establishing dense correspondence across object instances and transferring task-relevant grasps to novel instances. Extensive experiments on task-relevant grasping of densely-cluttered industrial objects are conducted in both simulation and real-world setups, demonstrating the effectiveness of the proposed framework.
more »
« less
A Parametric Grasping Methodology for Multi-Manual Interactions in Real-Time Dynamic Simulations
Interactive simulators are used in several important applications which include the training simulators for teleoperated robotic laparoscopic surgery. While stateof-art simulators are capable of rendering realistic visuals and accurate dynamics, grasping is often implemented using kinematic simplification techniques that prevent truly multimanual manipulation, which is often an important requirement of the actual task. Realistic grasping and manipulation in simulation is a challenging problem due to the constraints imposed by the implementation of rigid-body dynamics and collision computation techniques in state-of-the-art physics libraries. We present a penalty based parametric approach to achieve multi-manual grasping and manipulation of complex objects at arbitrary postures in a real-time dynamic simulation. This approach is demonstrated by accomplishing multi-manual tasks modeled after realistic scenarios, which include the grasping and manipulation of a two-handed screwdriver task and the manipulation of a deformable thread.
more »
« less
- PAR ID:
- 10207766
- Date Published:
- Journal Name:
- 2020 IEEE International Conference on Robotics and Automation (ICRA)
- Page Range / eLocation ID:
- 8712 to 8718
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In chart-based programming environments for machine knitting, patterns are specified at a low level by placing operations on a grid. This highly manual workflow makes it challenging to iterate on design elements such as cables, colorwork, and texture. While vector-based abstractions for knitting design elements may facilitate higher-level manipulation, they often include interdependencies which require stitch-level reconciliation. To address this, we contribute a new way of specifying knits with blended vector and raster primitives. Our abstraction supports the design of interdependent elements like colorwork and texture. We have implemented our blended raster/vector specification in a direct manipulation design tool where primitives are layered and rasterized, allowing for simulation of the resulting knit structure and generation of machine instructions. Through examples, we show how our approach enables higher-level manipulation of various knitting techniques, including intarsia colorwork, short rows, and cables. Specifically, we show how our tool supports the design of complex patterns including origami pleat patterns and capacitive sensor patches.more » « less
-
Promising results have been achieved recently in category-level manipulation that generalizes across object instances. Nevertheless, it often requires expensive real-world data collection and manual specification of semantic keypoints for each object category and task. Additionally, coarse keypoint predictions and ignoring intermediate action sequences hinder adoption in complex manipulation tasks beyond pick-and-place. This work proposes a novel, category-level manipulation framework that leverages an object-centric, category-level representation and model-free 6 DoF motion tracking. The canonical object representation is learned solely in simulation and then used to parse a category-level, task trajectory from a single demonstration video. The demonstration is reprojected to a target trajectory tailored to a novel object via the canonical representation. During execution, the manipulation horizon is decomposed into longrange, collision-free motion and last-inch manipulation. For the latter part, a category-level behavior cloning (CatBC) method leverages motion tracking to perform closed-loop control. CatBC follows the target trajectory, projected from the demonstration and anchored to a dynamically selected category-level coordinate frame. The frame is automatically selected along the manipulation horizon by a local attention mechanism. This framework allows to teach different manipulation strategies by solely providing a single demonstration, without complicated manual programming. Extensive experiments demonstrate its efficacy in a range of challenging industrial tasks in highprecision assembly, which involve learning complex, long-horizon policies. The process exhibits robustness against uncertainty due to dynamics as well as generalization across object instances and scene configurations.more » « less
-
Robot manipulation in cluttered environments of-ten requires complex and sequential rearrangement of multiple objects in order to achieve the desired reconfiguration of the target objects. Due to the sophisticated physical interactions involved in such scenarios, rearrangement-based manipulation is still limited to a small range of tasks and is especially vulnerable to physical uncertainties and perception noise. This paper presents a planning framework that leverages the efficiency of sampling-based planning approaches, and closes the manipulation loop by dynamically controlling the planning horizon. Our approach interleaves planning and execution to progressively approach the manipulation goal while correcting any errors or path deviations along the process. Meanwhile, our framework allows the definition of manipulation goals without requiring explicit goal configurations, enabling the robot to flexibly interact with all objects to facilitate the manipulation of the target ones. With extensive experiments both in simulation and on a real robot, we evaluate our framework on three manipulation tasks in cluttered environments: grasping, relocating, and sorting. In comparison with two baseline approaches, we show that our framework can significantly improve planning efficiency, robustness against physical uncertainties, and task success rate under limited time budgets.more » « less
-
While many robotic tasks, like manipulation and locomotion, are fundamentally based in making and breaking contact with the environment, state-of-the-art control policies struggle to deal with the hybrid nature of multi-contact motion. Such controllers often rely heavily upon heuristics or, due to the combinatoric structure in the dynamics, are unsuitable for real-time control. Principled deployment of tactile sensors offers a promising mechanism for stable and robust control, but modern approaches often use this data in an ad hoc manner, for instance to guide guarded moves. In this work, by exploiting the complementarity structure of contact dynamics, we propose a control framework which can close the loop on rich, tactile sensors. Critically, this framework is non-combinatoric, enabling optimization algorithms to automatically synthesize provably stable control policies. We demonstrate this approach on three different underactuated, multi-contact robotics problems.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ICRA40945.2020.9197099
