skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Reversible and irreversible bracket-based dynamics for deep graph neural networks
Recent works have shown that physics-inspired architectures allow the training of deep graph neural networks (GNNs) without oversmoothing. The role of these physics is unclear, however, with successful examples of both reversible (e.g., Hamiltonian) and irreversible (e.g., diffusion) phenomena producing comparable results despite diametrically opposed mechanisms, and further complications arising due to empirical departures from mathematical theory. This work presents a series of novel GNN architectures based upon structure preserving bracket-based dynamical systems, which are provably guaranteed to either conserve energy or generate positive dissipation with increasing depth. It is shown that the theoretically principled framework employed here allows for inherently explainable constructions, which contextualize departures from theory in current architectures and better elucidate the roles of reversibility and irreversibility in network performance. Code is available at the Github repository https://github.com/natrask/BracketGraphs.  more » « less
Award ID(s):
2210137
PAR ID:
10568040
Author(s) / Creator(s):
; ;
Publisher / Repository:
Advances in Neural Information Processing
Date Published:
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    Servo error pre-compensation (SEP) is commonly used to improve the accuracy of feed drives. Existing SEP approaches often involve the use of physics-based linear models (e.g., transfer functions) to predict servo errors, but suffer from inaccuracies due to unmodeled nonlinear dynamics in feed drives. This paper proposes a linear hybrid model for SEP that combines physics-based and data-driven linear models. The proposed model is shown to approximate nonlinearities unmodeled in physics-based linear models. In experiments on a precision feed drive, the proposed hybrid model improves the accuracy of servo error prediction by up to 38% compared to a physics-based model. 
    more » « less
  2. Nearly all animals forage to acquire energy for survival through efficient search and resource harvesting. Patch exploitation is a canonical foraging behaviour, but there is a need for more tractable and understandable mathematical models describing how foragers deal with uncertainty. To provide such a treatment, we develop a normative theory of patch foraging decisions, proposing mechanisms by which foraging behaviours emerge in the face of uncertainty. Our model foragers statistically and sequentially infer patch resource yields using Bayesian updating based on their resource encounter history. A decision to leave a patch is triggered when the certainty of the patch type or the estimated yield of the patch falls below a threshold. The time scale over which uncertainty in resource availability persists strongly impacts behavioural variables like patch residence times and decision rules determining patch departures. When patch depletion is slow, as in habitat selection, departures are characterized by a reduction of uncertainty, suggesting that the forager resides in a low-yielding patch. Uncertainty leads patch-exploiting foragers to overharvest (underharvest) patches with initially low (high) resource yields in comparison with predictions of the marginal value theorem. These results extend optimal foraging theory and motivate a variety of behavioural experiments investigating patch foraging behaviour. 
    more » « less
  3. The emergence of orbital resonances among planets is a natural consequence of the early dynamical evolution of planetary systems. While it is well established that convergent migration is necessary for mean-motion commensurabilities to emerge, recent numerical experiments have shown that the existing adiabatic theory of resonant capture provides an incomplete description of the relevant physics, leading to an erroneous mass scaling in the regime of strong dissipation. In this work, we develop a new model for resonance capture that self-consistently accounts for migration and circularization of planetary orbits, and derive an analytic criterion based upon stability analysis that describes the conditions necessary for the formation of mean-motion resonances. We subsequently test our results against numerical simulations and find satisfactory agreement. Our results elucidate the critical role played by adiabaticity and resonant stability in shaping the orbital architectures of planetary systems during the nebular epoch, and provide a valuable tool for understanding their primordial dynamical evolution. 
    more » « less
  4. We introduce the Discrete-Temporal Sobolev Network (DTSN), a neural network loss function that assists dynamical system forecasting by minimizing variational differences between the network output and the training data via a temporal Sobolev norm. This approach is entirely data-driven, architecture agnostic, and does not require derivative information from the estimated system. The DTSN is particularly well suited to chaotic dynamical systems as it minimizes noise in the network output which is crucial for such sensitive systems. For our test cases we consider discrete approximations of the Lorenz-63 system and the Chua circuit. For the network architectures we use the Long Short-Term Memory (LSTM) and the Transformer. The performance of the DTSN is compared with the standard MSE loss for both architectures, as well as with the Physics Informed Neural Network (PINN) loss for the LSTM. The DTSN loss is shown to substantially improve accuracy for both architectures, while requiring less information than the PINN and without noticeably increasing computational time, thereby demonstrating its potential to improve neural network forecasting of dynamical systems. 
    more » « less
  5. Demeniconi, Carlotta; Davidson, Ian (Ed.)
    This paper proposes a physics-guided machine learning approach that combines machine learning models and physics-based models to improve the prediction of water flow and temperature in river networks. We first build a recurrent graph network model to capture the interactions among multiple segments in the river network. Then we transfer knowledge from physics-based models to guide the learning of the machine learning model. We also propose a new loss function that balances the performance over different river segments. We demonstrate the effectiveness of the proposed method in predicting temperature and streamflow in a subset of the Delaware River Basin. In particular, the proposed method has brought a 33%/14% accuracy improvement over the state-of-the-art physics-based model and 24%/14% over traditional machine learning models (e.g., LSTM) in temperature/streamflow prediction using very sparse (0.1%) training data. The proposed method has also been shown to produce better performance when generalized to different seasons or river segments with different streamflow ranges. 
    more » « less