An official website of the United States government
We introduce the Discrete-Temporal Sobolev Network (DTSN), a neural network loss function that assists dynamical system forecasting by minimizing variational differences between the network output and the training data via a temporal Sobolev norm. This approach is entirely data-driven, architecture agnostic, and does not require derivative information from the estimated system. The DTSN is particularly well suited to chaotic dynamical systems as it minimizes noise in the network output which is crucial for such sensitive systems. For our test cases we consider discrete approximations of the Lorenz-63 system and the Chua circuit. For the network architectures we use the Long Short-Term Memory (LSTM) and the Transformer. The performance of the DTSN is compared with the standard MSE loss for both architectures, as well as with the Physics Informed Neural Network (PINN) loss for the LSTM. The DTSN loss is shown to substantially improve accuracy for both architectures, while requiring less information than the PINN and without noticeably increasing computational time, thereby demonstrating its potential to improve neural network forecasting of dynamical systems.
more »
« less
This content will become publicly available on July 8, 2026
Physics-Constrained Taylor Neural Networks for Learning and Control of Dynamical Systems
Data-driven approaches are increasingly popular for identifying dynamical systems due to improved accuracy and availability of sensor data. However, relying solely on data for identification does not guarantee that the identified systems will maintain their physical properties or that the predicted models will generalize well. In this paper, we propose a novel method for data-driven system identification by integrating a neural network as the first-order derivative of the learned dynamics in a Taylor series instead of learning the dynamical function directly. In addition, for dynamical systems with known monotonic properties, our approach can ensure monotonicity by constraining the neural network derivative to be non-positive or non-negative to the respective inputs, resulting in Monotonic Taylor Neural Networks (MTNN). Such constraints are enforced by either a specialized neural network architecture or regularization in the loss function for training. The proposed method demonstrates better performance compared to methods without the physics-based monotonicity constraints when tested on experimental data from an HVAC system and a temperature control testbed. Furthermore, MTNN shows good performance in a control application of a model predictive controller for a nonlinear MIMO system, illustrating the practical application of our method.
more »
« less
- PAR ID:
- 10573695
- Publisher / Repository:
- IEEE
- Date Published:
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This position paper introduces a Dynamic Data Driven Open Radio Access Network System (3D-O-RAN). The key objective of 3D-O-RAN is to support congested, contested and contaminated tactical settings where multimedia sensors, application constraints and operating wireless conditions may frequently change over space, time and frequency. 3D-O-RAN is compliant with the O-RAN specification for beyond 5G cellular systems to reduce costs and guarantee interoperability among vendors. Moreover, 3D-O-RAN integrates computational, sensing, and cellular networking components in a highly-dynamic, feedback-based, data-driven control loop. Specifically, 3D-O-RAN is designed to incorporate heterogeneous data into the network control loop to achieve a system-wide optimal operating point. Moreover, 3D-O-RAN steers the multimedia sensor measurement process in real time according to the required application needs and current physical and/or environmental constraints. 3D-O-RAN uses (i) a semantic slicing engine, which takes into account the semantic of the application to optimally compress the multimedia stream without losing in classification accuracy; (ii) a dynamic data driven neural network certification system that translates mission-level constraints into technical-level constraints on neural network latency/accuracy, and occupation of hardware/software resources. Realistic use-case scenarios of 3D-O-RAN in a tactical context demonstrate system performance.more » « less
-
Neural ordinary differential equations (NODEs) -- parametrizations of differential equations using neural networks -- have shown tremendous promise in learning models of unknown continuous-time dynamical systems from data. However, every forward evaluation of a NODE requires numerical integration of the neural network used to capture the system dynamics, making their training prohibitively expensive. Existing works rely on off-the-shelf adaptive step-size numerical integration schemes, which often require an excessive number of evaluations of the underlying dynamics network to obtain sufficient accuracy for training. By contrast, we accelerate the evaluation and the training of NODEs by proposing a data-driven approach to their numerical integration. The proposed Taylor-Lagrange NODEs (TL-NODEs) use a fixed-order Taylor expansion for numerical integration, while also learning to estimate the expansion's approximation error. As a result, the proposed approach achieves the same accuracy as adaptive step-size schemes while employing only low-order Taylor expansions, thus greatly reducing the computational cost necessary to integrate the NODE. A suite of numerical experiments, including modeling dynamical systems, image classification, and density estimation, demonstrate that TL-NODEs can be trained more than an order of magnitude faster than state-of-the-art approaches, without any loss in performance.more » « less
-
Identifying hidden interactions within complex systems is key to unlocking deeper insights into their operational dynamics, including how their elements affect each other and contribute to the overall system behavior. For instance, in neuroscience, discovering neuron-to-neuron interactions is essential for understanding brain function; in ecology, recognizing interactions among populations is key to understanding complex ecosystems. Such systems, often modeled as dynamical systems, typically exhibit noisy high-dimensional and non-stationary temporal behavior that renders their identification challenging. Existing dynamical system identification methods typically yield operators that accurately capture short-term behavior but fail to predict long-term trends, suggesting an incomplete capture of the underlying process. Methods that consider extended forecasts (e.g., recurrent neural networks) lack explicit representations of element-wise interactions and require substantial training data, thereby failing to capture interpretable network operators. Here we introduce Lookahead-driven Inference of Networked Operators for Continuous Stability (LINOCS), a robust learning procedure for identifying hidden dynamical interactions in noisy time-series data. LINOCS integrates several multi-step predictions with adaptive weights during training to recover dynamical operators that can yield accurate long-term predictions. We demonstrate LINOCS’ ability to recover the ground truth dynamical operators underlying synthetic time-series data for multiple dynamical systems models (including linear, piece-wise linear, time-changing linear systems’ decomposition, and regularized linear time-varying systems) as well as its capability to produce meaningful operators with robust reconstructions through various real-world examplesmore » « less
-
Firoozi, R.; Mehr, N.; Yel, E.; Antonova, R.; Bohg, J.; Schwager, M.; Kochenderfer, M. (Ed.)We develop a learning-based control algorithm for unknown dynamical systems under very severe data limitations. Specifically, the algorithm has access to streaming and noisy data only from a sin- gle and ongoing trial. It accomplishes such performance by effectively leveraging various forms of side information on the dynamics to reduce the sample complexity. Such side information typically comes from elementary laws of physics and qualitative properties of the system. More precisely, the algorithm approximately solves an optimal control problem encoding the system’s desired be- havior. To this end, it constructs and iteratively refines a data-driven differential inclusion that contains the unknown vector field of the dynamics. The differential inclusion, used in an interval Taylor-based method, enables to over-approximate the set of states the system may reach. Theo- retically, we establish a bound on the suboptimality of the approximate solution with respect to the optimal control with known dynamics. We show that the longer the trial or the more side infor- mation is available, the tighter the bound. Empirically, experiments in a high-fidelity F-16 aircraft simulator and MuJoCo’s environments illustrate that, despite the scarcity of data, the algorithm can provide performance comparable to reinforcement learning algorithms trained over millions of environment interactions. Besides, we show that the algorithm outperforms existing techniques combining system identification and model predictive control.more » « less
