An official website of the United States government
This work proposes a robotic pipeline for picking and constrained placement of objects without geometric shape priors. Compared to recent efforts developed for similar tasks, where every object was assumed to be novel, the proposed system recognizes previously manipulated objects and performs online model reconstruction and reuse. Over a lifelong manipulation process, the system keeps learning features of objects it has interacted with and updates their reconstructed models. Whenever an instance of a previously manipulated object reappears, the system aims to first recognize it and then register its previously reconstructed model given the current observation. This step greatly reduces object shape uncertainty allowing the system to even reason for parts of objects, which are currently not observable. This also results in better manipulation efficiency as it reduces the need for active perception of the target object during manipulation. To get a reusable reconstructed model, the proposed pipeline adopts: i) TSDF for object representation, and ii) a variant of the standard particle filter algorithm for pose estimation and tracking of the partial object model. Furthermore, an effective way to construct and maintain a dataset of manipulated objects is presented. A sequence of real-world manipulation experiments is performed. They show how future manipulation tasks become more effective and efficient by reusing reconstructed models of previously manipulated objects, which were generated during their prior manipulation, instead of treating objects as novel every time.
more »
« less
The Benefits of Near-field Manipulation and Viewing to Distant Object Manipulation in VR
In this contribution, we propose to enhance two distant object manipulation techniques, BMSR (Bimanual Near-Field Metaphor with Scaled Replica) and the classic Scaled HOMER (Scaled Hand-Centered Object Manipulation Extending Ray Casting), via nearfield scaled replica manipulation and viewing. In the proposed Direct BMSR, context replicas are displayed so that the target replica can be manipulated relative to its context, allowing the user to directly manipulate the target replica in their arm’s reach space. Some additional features were implemented to make Direct BMSR an effective interface for manipulating objects from a distance. We proposed Scaled HOMER+NFSRV, which augments Scaled HOMER with a near-field scaled replica view (NFSRV) of the target object and its context, enabling the user to observe how the target replica is manipulated in relation to its context in their arm’s reach space while manipulating it from a distance. We conducted a between-subjects empirical evaluation of BMSR, Direct BMSR, Scaled HOMER, and Scaled HOMER+NFSRV. Our findings revealed that Direct BMSR and Scaled HOMER+NFSRV significantly outperformed BMSR and Scaled HOMER, respectively, in terms of accuracy. This finding highlights the advantages of adding near-field scaled replica viewing and manipulation with respect to distant object manipulation.
more »
« less
- Award ID(s):
- 2007435
- PAR ID:
- 10527847
- Publisher / Repository:
- IEEE
- Date Published:
- ISSN:
- 2642-5254
- ISBN:
- 979-8-3503-7402-5
- Page Range / eLocation ID:
- 408 to 417
- Format(s):
- Medium: X
- Location:
- Orlando, FL, USA
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Humans demonstrate an impressive ability to acquire and generalize manipulation “tricks.” Even from a single demonstration, such as using soup ladles to reach for distant objects, we can apply this skill to new scenarios involving different object positions, sizes, and categories (e.g., forks and hammers). Addi- tionally, we can flexibly combine various skills to devise long-term plans. In this paper, we present a framework that enables machines to acquire such manipulation skills, referred to as “mechanisms,” through a single demonstration and self-play. Our key insight lies in interpreting each demonstration as a sequence of changes in robot-object and object-object contact modes, which provides a scaffold for learning detailed samplers for continuous parameters. These learned mechanisms and samplers can be seamlessly integrated into standard task and motion planners, enabling their compositional use.more » « less
-
This article extends recent work in magnetic manipulation of conductive, nonmagnetic objects using rotating magnetic dipole fields. Eddy-current-based manipulation provides a contact-free way to manipulate metallic objects. We are particularly motivated by the large amount of aluminum in space debris. We previously demonstrated dexterous manipulation of solid spheres with all object parameters known a priori. This work expands the previous model, which contained three discrete modes, to a continuous model that covers all possible relative positions of the manipulated spherical object with respect to the magnetic field source. We further leverage this new model to examine manipulation of spherical objects with unknown physical parameters by applying techniques from the online-optimization and adaptive-control literature. Our experimental results validate our new dynamics model, showing that we get improved performance compared to the previously proposed model, while also solving a simpler optimization problem for control. We further demonstrate the first physical magnetic manipulation of aluminum spheres, as previous controllers were only physically validated on copper spheres. We show that our adaptive control framework can quickly acquire useful object parameters when weakly initialized. Finally, we demonstrate that the spherical-object model can be used as an approximate model for adaptive control of nonspherical objects by performing magnetic manipulation of a variety of objects for which a spherical model is not an obvious approximation.more » « less
-
This paper studies dexterous manipulation in the plane by a two-fingered hand in the plane. The dynamics of each finger, which consists of two links with coupled joints, are derived based on Lagrangian mechanics. As an object is being manipulated, its orientation and the two independent joint angles of the hand constitute the state of the entire system. Contact kinematics, accounting for both stick and slip modes, are combined with dynamics to establish a dependence of the object's linear and angular accelerations on joint accelerations. This allows control of joint torques, under a proportional-derivative (PD) law, to move the object to a target position in a desired orientation.more » « less
-
This paper studies dexterous manipulation in the plane by a two-fingered hand in the plane. The dynamics of each finger, which consists of two links with coupled joints, are derived based on Lagrangian mechanics. As an object is being manipulated, its orientation and the two independent joint angles of the hand constitute the state of the entire system. Contact kinematics, accounting for both stick and slip modes, are combined with dynamics to establish a dependence of the object's linear and angular accelerations on joint accelerations. This allows control of joint torques, under a proportional-derivative (PD) law, to move the object to a target position in a desired orientation.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/VR58804.2024.00062
