skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Self-supervised Cloth Reconstruction via Action-conditioned Cloth Tracking
Award ID(s):
1849154 2046491
PAR ID:
10433356
Author(s) / Creator(s):
Date Published:
Journal Name:
arXivorg
ISSN:
2331-8422
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
  2. Robotic manipulation of cloth remains challenging due to the complex dynamics of cloth, lack of a low-dimensional state representation, and self-occlusions. In contrast to previous model-based approaches that learn a pixel-based dynamics model or a compressed latent vector dynamics, we propose to learn a particle-based dynamics model from a partial point cloud observation. To overcome the challenges of partial observability, we infer which visible points are connected on the underlying cloth mesh. We then learn a dynamics model over this visible connectivity graph. Compared to previous learning-based approaches, our model poses strong inductive bias with its particle based representation for learning the underlying cloth physics; it can generalize to cloths with novel shapes; it is invariant to visual features; and the predictions can be more easily visualized. We show that our method greatly outperforms previous state-of-the-art model-based and model-free reinforcement learning methods in simulation. Furthermore, we demonstrate zero-shot sim-to-real transfer where we deploy the model trained in simulation on a Franka arm and show that the model can successfully smooth cloths of different materials, geometries and colors from crumpled configurations. 
    more » « less
  3. null (Ed.)
    Cloth detection and manipulation is a common task in domestic and industrial settings, yet such tasks remain a challenge for robots due to cloth deformability. Furthermore, in many cloth-related tasks like laundry folding and bed making, it is crucial to manipulate specific regions like edges and corners, as opposed to folds. In this work, we focus on the problem of segmenting and grasping these key regions. Our approach trains a network to segment the edges and corners of a cloth from a depth image, distinguishing such regions from wrinkles or folds. We also provide a novel algorithm for estimating the grasp location, direction, and directional uncertainty from the segmentation. We demonstrate our method on a real robot system and show that it outperforms baseline methods on grasping success. 
    more » « less
  4. Identifying foreign cloth imports in the Icelandic archaeological corpus is difficult at best, yet given widespread similarities in homespun cloth from sites across the country, imported cloth can be identified visually through the presence of refined finishing techniques (such as teaseling, shearing, and fulling) that were uncommon in Iceland and were the products of specialist craftsmen in Europe. This paper examines textile assemblages from deposits datable to the period of Hanseatic trade at three sites, Gilsbakki, Reykholt, and Stóra-Borg that represent two wealthy, interior, parish centres and a moderate-sized coastal farm, respectively. Variations in the number and diversity of imported cloth items within these sites’ assemblages suggest that while Hanseatic material culture was widely spread on Icelandic rural sites, the nature of the material culture sub-assemblages attributable to Hanseatic trade was not obviously a direct function of households’ wealth or proximity to harbours but may have engaged other cultural factors linked to the political and social challenges of the post-Reformation period and the roles of individual households in regional or intra-Icelandic trade. 
    more » « less