More Like this

1. Reachability Analysis of a General Class of Neural Ordinary Differential Equations

   https://doi.org/10.1007/978-3-031-15839-1_15  

   Manzanas Lopez, D.; Musau, P.; Hamilton, N.P.; Johnson, T.T. (August 2022, Formal Modeling and Analysis of Timed Systems. FORMATS 2022)

   Bogomolov, S.; Parker, D. (Ed.)

   Continuous deep learning models, referred to as Neural Ordinary Differential Equations (Neural ODEs), have received considerable attention over the last several years. Despite their burgeoning impact, there is a lack of formal analysis techniques for these systems. In this paper, we consider a general class of neural ODEs with varying architectures and layers, and introduce a novel reachability framework that allows for the formal analysis of their behavior. The methods developed for the reachability analysis of neural ODEs are implemented in a new tool called NNVODE. Specifically, our work extends an existing neural network verification tool to support neural ODEs. We demonstrate the capabilities and efficacy of our methods through the analysis of a set of benchmarks that include neural ODEs used for classification, and in control and dynamical systems, including an evaluation of the efficacy and capabilities of our approach with respect to existing software tools within the continuous-time systems reachability literature, when it is possible to do so. 

   more » « less
2. Two-Stage Approach to Parameter Estimation of Differential Equations Using Neural ODEs

   https://doi.org/10.1021/acs.iecr.1c00552  

   Bradley, William; Boukouvala, Fani (November 2021, Industrial engineering chemistry research)

   Modeling physiochemical relationships using dynamic data is a common task in fields throughout science and engineering. A common step in developing generalizable, mechanistic models is to fit unmeasured parameters to measured data. However, fitting differential equation-based models can be computation-intensive and uncertain due to the presence of nonlinearity, noise, and sparsity in the data, which in turn causes convergence to local minima and divergence issues. This work proposes a merger of machine learning (ML) and mechanistic approaches by employing ML models as a means to fit nonlinear mechanistic ordinary differential equations (ODEs). Using a two-stage indirect approach, neural ODEs are used to estimate state derivatives, which are then used to estimate the parameters of a more interpretable mechanistic ODE model. In addition to its computational efficiency, the proposed method demonstrates the ability of neural ODEs to better estimate derivative information than interpolating methods based on algebraic data-driven models. Most notably, the proposed method is shown to yield accurate predictions even when little information is known about the parameters of the ODEs. The proposed parameter estimation approach is believed to be most advantageous when the ODE to be fit is strongly nonlinear with respect to its unknown parameters. 

   more » « less
3. Taylor-Model Physics-Informed Neural Networks (PINNs) for Ordinary Differential Equations

   Nagesh, Chandra Kanth; Sankaranarayanan, Sriram; Kaur, Ramneet; Sahai, Tuhin; Jha, Susmit (May 2025, Proceedings of Machine Learning Research)

   Pappas, George; Ravikumar, Pradeep; Seshia, Sanjit A (Ed.)

   We study the problem of learning neural network models for Ordinary Differential Equations (ODEs) with parametric uncertainties. Such neural network models capture the solution to the ODE over a given set of parameters, initial conditions, and range of times. Physics-Informed Neural Networks (PINNs) have emerged as a promising approach for learning such models that combine data-driven deep learning with symbolic physics models in a principled manner. However, the accuracy of PINNs degrade when they are used to solve an entire family of initial value problems characterized by varying parameters and initial conditions. In this paper, we combine symbolic differentiation and Taylor series methods to propose a class of higher-order models for capturing the solutions to ODEs. These models combine neural networks and symbolic terms: they use higher order Lie derivatives and a Taylor series expansion obtained symbolically, with the remainder term modeled as a neural network. The key insight is that the remainder term can itself be modeled as a solution to a first-order ODE. We show how the use of these higher order PINNs can improve accuracy using interesting, but challenging ODE benchmarks. We also show that the resulting model can be quite useful for situations such as controlling uncertain physical systems modeled as ODEs. 

   more » « less
4. Synthetic PMU Data Creation Based on Generative Adversarial Network under Time-Varying Load Conditions

   https://doi.org/10.35833/MPCE.2021.000783  

   Zheng, Xiangtian; Pinceti, Andrea; Sankar, Lalitha; Xie, Le (July 2022, Journal of modern power systems and clean energy)

   In this study, a machine learning based method is proposed for creating synthetic eventful phasor measurement unit (PMU) data under time-varying load conditions. The proposed method leverages generative adversarial networks to create quasi-steady states for the power system under slowly-varying load conditions and incorporates a framework of neural ordinary differential equations (ODEs) to capture the transient behaviors of the system during voltage oscillation events. A numerical example of a large power grid suggests that this method can create realistic synthetic eventful PMU voltage measurements based on the associated real PMU data without any knowledge of the underlying nonlinear dynamic equations. The results demonstrate that the synthetic voltage measurements have the key characteristics of real system behavior on distinct time scales. 

   more » « less
5. Enhancing Urban Flow Maps via Neural ODEs

   https://doi.org/10.24963/ijcai.2020/180  

   Zhou, Fan; Li, Liang; Zhong, Ting; Trajcevski, Goce; Zhang, Kunpeng; Wang, Jiahao (July 2020, Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, {IJCAI} 2020)

   null (Ed.)

   Flow super-resolution (FSR) enables inferring fine-grained urban flows with coarse-grained observations and plays an important role in traffic monitoring and prediction. The existing FSR solutions rely on deep CNN models (e.g., ResNet) for learning spatial correlation, incurring excessive memory cost and numerous parameter updates. We propose to tackle the urban flows inference using dynamic systems paradigm and present a new method FODE -- FSR with Ordinary Differential Equations (ODEs). FODE extends neural ODEs by introducing an affine coupling layer to overcome the problem of numerically unstable gradient computation, which allows more accurate and efficient spatial correlation estimation, without extra memory cost. In addition, FODE provides a flexible balance between flow inference accuracy and computational efficiency. A FODE-based augmented normalization mechanism is further introduced to constrain the flow distribution with the influence of external factors. Experimental evaluations on two real-world datasets demonstrate that FODE significantly outperforms several baseline approaches. 

   more » « less