An official website of the United States government
In acquiring a syntax, children must detect evidence for abstract structural dependencies that can be realized in variable ways in the surface forms of sentences. InWhat did David fix?, learners must identify a nonlocal relation between a fronted object of the verb (what) and the phonologically null ‘gap’ in canonical direct object position after the verb, where it is thematically interpreted. How do learners identify a nonadjacent dependency between an expression and something that has no overt phonological form? We propose that identifying abstract syntactic dependencies requires statistical inference over both overt linguistic material and unsatisfied grammatical expectations: noticing when a predicted argument for a verb is unexpectedly missing may serve as evidence for the gap of an argument movement dependency. We provide computational support for this hypothesis. We develop a learner that uses predicted but unexpectedly missing objects of verbs to identify possible gaps of object movement, and identifies which surface morphosyntactic properties of sentences are correlated with these possible movement gaps. We find that it is in principle possible for a learner using this mechanism to identify the majority of sentences with object movement in child-directed English, and that prior knowledge of which verbs require objects provides an important guide for identifying which surface distributions characterize object movement. This provides a computational account for why verb argument-structure knowledge developmentally precedes the acquisition of movement in a language like English. More broadly, these findings illustrate how statistical learning and learning from violated expectations can be combined to novel effect in the domain of language acquisition.
more »
« less
Skill Generalization with Verbs
It is imperative that robots can understand natural language commands issued by humans. Such commands typically contain verbs that signify what action should be performed on a given object and that are applicable to many objects. We propose a method for generalizing manipulation skills to novel objects using verbs. Our method learns a probabilistic classifier that determines whether a given object trajectory can be described by a specific verb. We show that this classifier accurately generalizes to novel object categories with an average accuracy of 76.69% across 13 object categories and 14 verbs. We then perform policy search over the object kinematics to find an object trajectory that maximizes classifier prediction for a given verb. Our method allows a robot to generate a trajectory for a novel object based on a verb, which can then be used as input to a motion planner. We show that our model can generate trajectories that are usable for executing five verb commands applied to novel instances of two different object categories on a real robot.
more »
« less
- Award ID(s):
- 1955361
- PAR ID:
- 10467322
- Publisher / Repository:
- Proceedings of the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems
- Date Published:
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
ABSTRACT The ability to represent both local and nonlocal syntactic dependencies emerges in an infant's second year of life, raising questions about how these early syntactic representations interact with language learning in other domains. Usingwh‐questions as our case study, we investigate how infants’ syntactic dependency acquisition interacts with their early lexical development. Prior work finds that 18‐month‐olds represent frontedwh‐phrases as nonlocal arguments in objectwh‐questions with known verbs. Here, we show that 19–21‐month‐olds (range: 18;29–21;26) do the same when interpreting unknown verbs. We introduce a novel Violation of Fit method, a cross‐modal extension of the Violation of Expectations paradigm. Infants saw dialogues with novel verbs in objectwh‐questions (e.g.,What is the girl gonna gorp?), transitive polar questions (Is the girl gonna gorp the toy?), or intransitive polar questions(Is the girl gonna gorp?). At test, infants viewed a causal event (e.g., a girl knocks over a tower), and we measured their attention as an indication of whether they considered the verbs to be a good fit for this type of event. Across the age range, we found that infants who heardwh‐question dialogues attended similarly to the test events as infants who heard canonical transitive dialogues, and unlike infants who heard intransitive dialogues. Thus, 19–21‐month‐olds treat objectwh‐questions with a novel verb as transitive when relating them to scenes. This suggests that immediately afterwh‐dependency representations are first acquired, they are available to feed verb learning. SummaryWe find that 19–21‐month‐olds represent wh‐phrases (e.g.,what) as nonlocal objects in wh‐questions with unknown verbs (e.g.,What is the girl gonna gorp?).We introduce a novel experimental paradigm, Violation of Fit, which measures how well an infant considers a particular sentence to fit with a particular scene.This test reveals that infants treat object wh‐questions with a novel verb as transitive, and therefore a good fit to a causal event.This suggests that immediately after wh‐dependency representations are first acquired, they are available to feed verb learning.more » « less
-
null (Ed.)Robot manipulation and grasping mechanisms have received considerable attention in the recent past, leading to development of wide-range of industrial applications. This paper proposes the development of an autonomous robotic grasping system for object sorting application. RGB-D data is used by the robot for performing object detection, pose estimation, trajectory generation and object sorting tasks. The proposed approach can also handle grasping on certain objects chosen by users. Trained convolutional neural networks are used to perform object detection and determine the corresponding point cloud cluster of the object to be grasped. From the selected point cloud data, a grasp generator algorithm outputs potential grasps. A grasp filter then scores these potential grasps, and the highest-scored grasp will be chosen to execute on a real robot. A motion planner will generate collision-free trajectories to execute the chosen grasp. The experiments on AUBO robotic manipulator show the potentials of the proposed approach in the context of autonomous object sorting with robust and fast sorting performance.more » « less
-
null (Ed.)As autonomous robots interact and navigate around real-world environments such as homes, it is useful to reliably identify and manipulate articulated objects, such as doors and cabinets. Many prior works in object articulation identification require manipulation of the object, either by the robot or a human. While recent works have addressed predicting articulation types from visual observations alone, they often assume prior knowledge of category-level kinematic motion models or sequence of observations where the articulated parts are moving according to their kinematic constraints. In this work, we propose FormNet, a neural network that identifies the articulation mechanisms between pairs of object parts from a single frame of an RGB-D image and segmentation masks. The network is trained on 100k synthetic images of 149 articulated objects from 6 categories. Synthetic images are rendered via a photorealistic simulator with domain randomization. Our proposed model predicts motion residual flows of object parts, and these flows are used to determine the articulation type and parameters. The network achieves an articulation type classification accuracy of 82.5% on novel object instances in trained categories. Experiments also show how this method enables generalization to novel categories and can be applied to real-world images without fine-tuning.more » « less
-
null (Ed.)bot manipulation and grasping mechanisms have received considerable attention in the recent past, leading to development of wide-range of industrial applications. This paper proposes the development of an autonomous robotic grasping system for object sorting application. RGB-D data is used by the robot for performing object detection, pose estimation, trajectory generation and object sorting tasks. The proposed approach can also handle grasping on certain objects chosen by users. Trained convolutional neural networks are used to perform object detection and determine the corresponding point cloud cluster of the object to be grasped. From the selected point cloud data, a grasp generator algorithm outputs potential grasps. A grasp filter then scores these potential grasps, and the highest-scored grasp will be chosen to execute on a real robot. A motion planner will generate collision-free trajectories to execute the chosen grasp. The experiments on AUBO robotic manipulator show the potentials of the proposed approach in the context of autonomous object sorting with robust and fast sorting performance.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Conference Paper:
- The DOI is not currently available.
