More Like this

1. Chop & Learn: Recognizing and Generating Object-State Compositions

   https://doi.org/10.1109/ICCV51070.2023.01852  

   Saini, Nirat ; Wang, Hanyu ; Swaminathan, Archana ; Jayasundara, Vinoj ; He, Bo ; Gupta, Kamal ; Shrivastava, Abhinav ( October 2023 , Proceedings of ICCV)

   Recognizing and generating object-state compositions has been a challenging task, especially when generalizing to unseen compositions. In this paper, we study the task of cutting objects in different styles and the resulting object state changes. We propose a new benchmark suite Chop & Learn, to accommodate the needs of learning objects and different cut styles using multiple viewpoints. We also propose a new task of Compositional Image Generation, which can transfer learned cut styles to different objects, by generating novel object-state images. Moreover, we also use the videos for Compositional Action Recognition, and show valuable uses of this dataset for multiple video tasks. Project website: https://chopnlearn.github.io. 

   more » « less
2. Learning Portable Representations for High-Level Planning

   James, S ; Rosman, B ; Konidaris, GD ( July 2020 , Proceedings of the Thirty-Seventh International Conference on Machine Learning)

   null (Ed.)

   We present a framework for autonomously learning a portable representation that describes a collection of low-level continuous environments. We show that these abstract representations can be learned in a task-independent egocentric space specific to the agent that, when grounded with problem-specific information, are provably sufficient for planning. We demonstrate transfer in two different domains, where an agent learns a portable, task-independent symbolic vocabulary, as well as operators expressed in that vocabulary, and then learns to instantiate those operators on a per-task basis. This reduces the number of samples required to learn a representation of a new task. 

   more » « less
3. Learning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulation

   Sharma, M. ; Liang, J. ; Zhao, J. ; LaGrassa, A. ; Kroemer, O. ( October 2020 , Conference on Robot Learning)

   null (Ed.)

   Manipulation tasks can often be decomposed into multiple subtasks performed in parallel, e.g., sliding an object to a goal pose while maintaining con- tact with a table. Individual subtasks can be achieved by task-axis controllers defined relative to the objects being manipulated, and a set of object-centric controllers can be combined in an hierarchy. In prior works, such combinations are defined manually or learned from demonstrations. By contrast, we propose using reinforcement learning to dynamically compose hierarchical object-centric controllers for manipulation tasks. Experiments in both simulation and real world show how the proposed approach leads to improved sample efficiency, zero-shot generalization to novel test environments, and simulation-to-reality transfer with- out fine-tuning. 

   more » « less
4. Hyperbolic Contrastive Learning for Visual Representations beyond Objects

   https://doi.org/10.1109/CVPR52729.2023.00661  

   Ge, Songwei ; Mishra, Shlok ; Kornblith, Simon ; Li, Chun-Liang ; Jacobs, David ( June 2023 , IEEE)

   Although self-/un-supervised methods have led to rapid progress in visual representation learning, these methods generally treat objects and scenes using the same lens. In this paper, we focus on learning representations for objects and scenes that preserve the structure among them. Motivated by the observation that visually similar objects are close in the representation space, we argue that the scenes and objects should instead follow a hierarchical structure based on their compositionality. To exploit such a structure, we propose a contrastive learning framework where a Euclidean loss is used to learn object representations and a hyperbolic loss is used to encourage representations of scenes to lie close to representations of their constituent objects in a hyperbolic space. This novel hyperbolic objective encourages the scene-object hypernymy among the representations by optimizing the magnitude of their norms. We show that when pretraining on the COCO and OpenImages datasets, the hyperbolic loss improves downstream performance of several baselines across multiple datasets and tasks, including image classification, object detection, and semantic segmentation. We also show that the properties of the learned representations allow us to solve various vision tasks that involve the interaction between scenes and objects in a zero-shot fashion. 

   more » « less
5. Policy Caches with Successor Features

   Nemecek, Mark ; Parr, Ronald ( July 2021 , Proceedings of Machine Learning Research)

   Meila, Marina ; Zhang, Tong (Ed.)

   Transfer in reinforcement learning is based on the idea that it is possible to use what is learned in one task to improve the learning process in another task. For transfer between tasks which share transition dynamics but differ in reward function, successor features have been shown to be a useful representation which allows for efficient computation of action-value functions for previously-learned policies in new tasks. These functions induce policies in the new tasks, so an agent may not need to learn a new policy for each new task it encounters, especially if it is allowed some amount of suboptimality in those tasks. We present new bounds for the performance of optimal policies in a new task, as well as an approach to use these bounds to decide, when presented with a new task, whether to use cached policies or learn a new policy. 

   more » « less