An official website of the United States government
Systems consisting of interacting agents are prevalent in the world, ranging from dynamical systems in physics to complex biological networks. To build systems which can interact robustly in the real world, it is thus important to be able to infer the precise interactions governing such systems. Existing approaches typically dis- cover such interactions by explicitly modeling the feed-forward dynamics of the trajectories. In this work, we propose Neural Interaction Inference with Potentials (NIIP) as an alternative approach to discover such interactions that enables greater flexibility in trajectory modeling: it discovers a set of relational potentials, represented as energy functions, which when minimized reconstruct the original trajectory. NIIP assigns low energy to the subset of trajectories which respect the relational constraints observed. We illustrate that with these representations NIIP displays unique capabilities in test-time. First, it allows trajectory manipulation, such as interchanging interaction types across separately trained models, as well as trajectory forecasting. Additionally, it allows adding external hand-crafted potentials at test-time. Finally, NIIP enables the detection of out-of-distribution samples and anomalies without explicit training.
more »
« less
Inferring Relational Potentials in Interacting Systems
Systems consisting of interacting agents are prevalent in the world, ranging from dynamical systems in physics to complex biological networks. To build systems which can interact robustly in the real world, it is thus important to be able to infer the precise interactions governing such systems. Existing approaches typically discover such interactions by explicitly modeling the feed-forward dynamics of the trajectories. In this work, we propose Neural Interaction Inference with Potentials (NIIP) as an alternative approach to discover such interactions that enables greater flexibility in trajectory modeling: it discovers a set of relational potentials, represented as energy functions, which when minimized reconstruct the original trajectory. NIIP assigns low energy to the subset of trajectories which respect the relational constraints observed. We illustrate that with these representations NIIP displays unique capabilities in test-time. First, it allows trajectory manipulation, such as interchanging interaction types across separately trained models, as well as trajectory forecasting. Additionally, it allows adding external hand-crafted potentials at test-time. Finally, NIIP enables the detection of out-of-distribution samples and anomalies without explicit training.
more »
« less
- Award ID(s):
- 2038493
- PAR ID:
- 10466333
- Date Published:
- Journal Name:
- Proceedings of Machine Learning Research
- ISSN:
- 2640-3498
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Understanding interactions between entities, e.g., joints of the human body, team sports players, etc., is crucial for tasks like forecasting. However, interactions between entities are commonly not observed and often hard to quantify. To address this challenge, recently, ‘Neural Relational Inference’ was introduced. It predicts static relations between entities in a system and provides an interpretable representation of the underlying system dynamics that are used for better trajectory forecasting. However, generally, relations between entities change as time progresses. Hence, static relations improperly model the data. In response to this, we develop Dynamic Neural Relational Inference (dNRI), which incorporates insights from sequential latent variable models to predict separate relation graphs for every time-step. We demonstrate on several real-world datasets that modeling dynamic relations improves forecasting of complex trajectories.more » « less
-
Abstract Many-body interactions between polymer-grafted nanoparticles (NPs) play a key role in promoting their assembly into low-dimensional structures within polymer melts, even when the particles are spherical and isotropically grafted. However, capturing such interactions in simulations of NP assembly is very challenging because explicit modeling of the polymer grafts and melt chains is highly computationally expensive, even using coarse-grained models. Here, we develop a many-body potential for describing the effective interactions between spherical polymer-grafted NPs in a polymer matrix through a machine-learning approach. The approach involves using permutationally invariant polynomials to fit two- and three-body interactions derived from the potential of mean force calculations. The potential developed here reduces the computational cost by several orders of magnitude, thereby, allowing us to explore assembly behavior over large length and time scales. We show that the potential not only reproduces previously known assembled phases such as 1D strings and 2D hexagonal sheets, which generally cannot be achieved using isotropic two-body potentials, but can also help discover interesting phases such as networks, clusters, and gels. We demonstrate how each of these assembly morphologies intrinsically arises from a competition between two- and three-body interactions. Our approach for deriving many-body effective potentials can be readily extended to other colloidal systems, enabling researchers to make accurate predictions of their behavior and dissect the role of individual interaction energy terms of the overall potential in the observed behavior.more » « less
-
Illegal vehicle parking is a common urban problem faced by major cities in the world, as it incurs traffic jams, which lead to air pollution and traffic accidents. Traditional approaches to detect illegal parking events rely highly on active human efforts. However, these approaches are extremely ineffective to cover a large city. The massive and high quality sharing bike trajectories from Mo- bike offer us with a unique opportunity to design a ubiquitous illegal parking detection system, as most of the illegal parking events happen at curbsides and have significant impact on the bike users. Two main components are employed in the proposed illegal park- ing detection system: 1) trajectory pre-processing, which filters outlier GPS points, performs map-matching and builds trajectory indexes; and 2) illegal parking detection, which models the normal trajectories, extracts features from the evaluation trajectories and utilizes a distribution test-based method to discover the illegal parking events. The system is deployed on the cloud, and used by Mo- bike internally. Finally, extensive experiments and many insightful case studies are presented.more » « less
-
We tackle the novel problem of predicting 3D hand motion and contact maps (or Interaction Trajectories) given a single RGB view, action text, and a 3D contact point on the object as input. Our approach consists of (1) Interaction Codebook: a VQVAE model to learn a latent codebook of hand poses and contact points, effectively tokenizing interaction trajectories, (2) Interaction Predictor: a transformer-decoder module to predict the interaction trajectory from test time inputs by using an indexer module to retrieve a latent affordance from the learned codebook. To train our model, we develop a data engine that extracts 3D hand poses and contact trajectories from the diverse HoloAssist dataset. We evaluate our model on a benchmark that is 2.5-10X larger than existing works, in terms of diversity of objects and interactions observed, and test for generalization of the model across object categories, action categories, tasks, and scenes. Experimental results show the effectiveness of our approach over transformer & diffusion baselines across all settings.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- The DOI is not currently available.
