More Like this

1. A Deep Learning-Based Autonomous Robot Manipulator for Sorting Application

   https://doi.org/10.1109/IRC.2020.00055  

   Bui, Hoang-Dung; Nguyen, Hai; La, Hung Manh; Li, Shuai (November 2020, 2020 Fourth IEEE International Conference on Robotic Computing (IRC))

   null (Ed.)

   Robot manipulation and grasping mechanisms have received considerable attention in the recent past, leading to development of wide-range of industrial applications. This paper proposes the development of an autonomous robotic grasping system for object sorting application. RGB-D data is used by the robot for performing object detection, pose estimation, trajectory generation and object sorting tasks. The proposed approach can also handle grasping on certain objects chosen by users. Trained convolutional neural networks are used to perform object detection and determine the corresponding point cloud cluster of the object to be grasped. From the selected point cloud data, a grasp generator algorithm outputs potential grasps. A grasp filter then scores these potential grasps, and the highest-scored grasp will be chosen to execute on a real robot. A motion planner will generate collision-free trajectories to execute the chosen grasp. The experiments on AUBO robotic manipulator show the potentials of the proposed approach in the context of autonomous object sorting with robust and fast sorting performance. 

   more » « less
2. Visual Identification of Articulated Object Parts

   Zeng, V.; Lee, T.; Liang, J.; Kroemer, O. (October 2021, Proceedings of the IEEERSJ International Conference on Intelligent Robots and Systems)

   null (Ed.)

   As autonomous robots interact and navigate around real-world environments such as homes, it is useful to reliably identify and manipulate articulated objects, such as doors and cabinets. Many prior works in object articulation identification require manipulation of the object, either by the robot or a human. While recent works have addressed predicting articulation types from visual observations alone, they often assume prior knowledge of category-level kinematic motion models or sequence of observations where the articulated parts are moving according to their kinematic constraints. In this work, we propose FormNet, a neural network that identifies the articulation mechanisms between pairs of object parts from a single frame of an RGB-D image and segmentation masks. The network is trained on 100k synthetic images of 149 articulated objects from 6 categories. Synthetic images are rendered via a photorealistic simulator with domain randomization. Our proposed model predicts motion residual flows of object parts, and these flows are used to determine the articulation type and parameters. The network achieves an articulation type classification accuracy of 82.5% on novel object instances in trained categories. Experiments also show how this method enables generalization to novel categories and can be applied to real-world images without fine-tuning. 

   more » « less
3. Mediating functions and the semantics of noun incorporation

   https://doi.org/10.1007/s11049-021-09511-4  

   McKenzie, Andrew (April 2021, Natural Language & Linguistic Theory)

   null (Ed.)

   Abstract Noun incorporation is commonly thought to avoid the weak compositionality of compounds because it involves conjunction of an argument noun with the incorporating verb. However, it is weakly compositional in two ways. First, the noun’s entity argument needs to be bound or saturated, but previous accounts fail to adequately ensure that it is. Second, non-arguments are often incorporated in many languages, and their thematic role is available for contextual selection. We show that these two weaknesses are actually linked. We focus on the Kiowa language, which generally bars objects from incorporation but allows non-arguments. We show that a mediating relation is required to semantically link the noun to the verb. Absent a relation, the noun’s entity argument is not saturated, and the entire expression is uninterpretable. The mediating relation for non-objects also assigns it a thematic role instead of a postposition. Speakers can choose this role freely, subject to independent constraints from the pragmatics, syntax, and semantics. Objects in Kiowa are in fact allowed to incorporate in certain environments, but we show that these all independently involve a mediating relation. The mediating relation for objects quantifies over the noun and links the noun+verb construction to the rest of the clause. The head that introduces this relation re-categorizes the verb in the syntactic derivation. Essentially, we demonstrate two distinct mechanisms for noun incorporation. Having derived the distribution of Kiowa, we apply the same relations to derive constraints on English complex verbs and synthetic compounds, which exhibit most of the same constraints as Kiowa noun incorporation. We also look at languages with routine object incorporation, and show how the transitivity of the verb depends on whether the v° head introducing the external argument assigns case to the re-categorized verb. 

   more » « less
4. A Deep Learning-Based Autonomous Robot Manipulator for Sorting Application

   https://doi.org/DOI 10.1109/IRC.2020.00055  

   Bui, Hoang-Dung; Nguyen, Hai; La, Hung M; Li, Shuai (November 2020, 2020 Fourth IEEE International Conference on Robotic Computing)

   null (Ed.)

   bot manipulation and grasping mechanisms have received considerable attention in the recent past, leading to development of wide-range of industrial applications. This paper proposes the development of an autonomous robotic grasping system for object sorting application. RGB-D data is used by the robot for performing object detection, pose estimation, trajectory generation and object sorting tasks. The proposed approach can also handle grasping on certain objects chosen by users. Trained convolutional neural networks are used to perform object detection and determine the corresponding point cloud cluster of the object to be grasped. From the selected point cloud data, a grasp generator algorithm outputs potential grasps. A grasp filter then scores these potential grasps, and the highest-scored grasp will be chosen to execute on a real robot. A motion planner will generate collision-free trajectories to execute the chosen grasp. The experiments on AUBO robotic manipulator show the potentials of the proposed approach in the context of autonomous object sorting with robust and fast sorting performance. 

   more » « less
5. Local Neural Descriptor Fields: Locally Conditioned Object Representations for Manipulation

   https://doi.org/10.1109/ICRA48891.2023.10160423  

   Chun, Ethan; Du, Yilun; Simeonov, Anthony; Lozano-Perez, Tomas; Kaelbling, Leslie (January 2023, IEEE International Conference on Robotics and Automation)

   A robot operating in a household environment will see a wide range of unique and unfamiliar objects. While a system could train on many of these, it is infeasible to predict all the objects a robot will see. In this paper, we present a method to generalize object manipulation skills acquired from a limited number of demonstrations, to novel objects from unseen shape categories. Our approach, Local Neural Descriptor Fields (LNDF), utilizes neural descriptors defined on the local geometry of the object to effectively transfer manipulation demonstrations to novel objects at test time. In doing so, we leverage the local geometry shared between objects to produce a more general manipulation framework. We illustrate the efficacy of our approach in manipulating novel objects in novel poses – both in simulation and in the real world. 

   more » « less