skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Two-Stage Approach to Parameter Estimation of Differential Equations Using Neural ODEs
Modeling physiochemical relationships using dynamic data is a common task in fields throughout science and engineering. A common step in developing generalizable, mechanistic models is to fit unmeasured parameters to measured data. However, fitting differential equation-based models can be computation-intensive and uncertain due to the presence of nonlinearity, noise, and sparsity in the data, which in turn causes convergence to local minima and divergence issues. This work proposes a merger of machine learning (ML) and mechanistic approaches by employing ML models as a means to fit nonlinear mechanistic ordinary differential equations (ODEs). Using a two-stage indirect approach, neural ODEs are used to estimate state derivatives, which are then used to estimate the parameters of a more interpretable mechanistic ODE model. In addition to its computational efficiency, the proposed method demonstrates the ability of neural ODEs to better estimate derivative information than interpolating methods based on algebraic data-driven models. Most notably, the proposed method is shown to yield accurate predictions even when little information is known about the parameters of the ODEs. The proposed parameter estimation approach is believed to be most advantageous when the ODE to be fit is strongly nonlinear with respect to its unknown parameters.  more » « less
Award ID(s):
1944678
PAR ID:
10334686
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Industrial engineering chemistry research
Volume:
60
Issue:
45
ISSN:
1520-5045
Page Range / eLocation ID:
16330–16344
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; ; ; ; (, IEEE Transactions on Biomedical Engineering)
    This paper proposes a method to learn ap- proximations of missing Ordinary Differential Equations (ODEs) and states in physiological models where knowl- edge of the system’s relevant states and dynamics is in- complete. The proposed method augments known ODEs with neural networks (NN), then trains the hybrid ODE-NN model on a subset of available physiological measurements (i.e., states) to learn the NN parameters that approximate the unknown ODEs. Thus, this method can model an ap- proximation of the original partially specified system sub- ject to the constraints of known biophysics. This method also addresses the challenge of jointly estimating physio- logical states, NN parameters, and unknown initial condi- tions during training using recursive Bayesian estimation. We validate this method using two simulated physiolog- ical systems, where subsets of the ODEs are assumed to be unknown during the training and test processes. The proposed method almost perfectly tracks the ground truth in the case of a single missing ODE and state and performs well in other cases where more ODEs and states are missing. This performance is robust to input signal per- turbations and noisy measurements. A critical advantage of the proposed hybrid methodology over purely data-driven methods is the incorporation of the ODE structure in the model, which allows one to infer unobserved physiological states. The ability to flexibly approximate missing or inac- curate components in ODE models improves a significant modeling bottleneck without sacrificing interpretability. 
    more » « less
  2. (, Advances in neural information processing systems)
    It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE). This observation motivated the introduction of so-called Neural ODEs, which allow more general discretization schemes with adaptive time stepping. Here, we propose ANODEV2, which is an extension of this approach that allows evolution of the neural network parameters, in a coupled ODE-based formulation. The Neural ODE method introduced earlier is in fact a special case of this new framework. We present the formulation of ANODEV2, derive optimality conditions, and implement the coupled framework in PyTorch. We present empirical results using several different configurations of ANODEV2, testing them on multiple models on CIFAR-10. We report results showing that this coupled ODE-based framework is indeed trainable, and that it achieves higher accuracy, as compared to the baseline models as well as the recently-proposed Neural ODE approach. 
    more » « less
  3. ; ; ; ; (, Proceedings of Machine Learning Research)
    Pappas, George; Ravikumar, Pradeep; Seshia, Sanjit A (Ed.)
    We study the problem of learning neural network models for Ordinary Differential Equations (ODEs) with parametric uncertainties. Such neural network models capture the solution to the ODE over a given set of parameters, initial conditions, and range of times. Physics-Informed Neural Networks (PINNs) have emerged as a promising approach for learning such models that combine data-driven deep learning with symbolic physics models in a principled manner. However, the accuracy of PINNs degrade when they are used to solve an entire family of initial value problems characterized by varying parameters and initial conditions. In this paper, we combine symbolic differentiation and Taylor series methods to propose a class of higher-order models for capturing the solutions to ODEs. These models combine neural networks and symbolic terms: they use higher order Lie derivatives and a Taylor series expansion obtained symbolically, with the remainder term modeled as a neural network. The key insight is that the remainder term can itself be modeled as a solution to a first-order ODE. We show how the use of these higher order PINNs can improve accuracy using interesting, but challenging ODE benchmarks. We also show that the resulting model can be quite useful for situations such as controlling uncertain physical systems modeled as ODEs. 
    more » « less
  4. ; ; (, Statistics in Medicine)
    Mathematical models based on systems of ordinary differential equations (ODEs) are frequently applied in various scientific fields to assess hypotheses, estimate key model parameters, and generate predictions about the system's state. To support their application, we present a comprehensive, easy‐to‐use, and flexible MATLAB toolbox,QuantDiffForecast, and associated tutorial to estimate parameters and generate short‐term forecasts with quantified uncertainty from dynamical models based on systems of ODEs. We provide software (https://github.com/gchowell/paramEstimation_forecasting_ODEmodels/) and detailed guidance on estimating parameters and forecasting time‐series trajectories that are characterized using ODEs with quantified uncertainty through a parametric bootstrapping approach. It includes functions that allow the user to infer model parameters and assess forecasting performance for different ODE models specified by the user, using different estimation methods and error structures in the data. The tutorial is intended for a diverse audience, including students training in dynamic systems, and will be broadly applicable to estimate parameters and generate forecasts from models based on ODEs. The functions included in the toolbox are illustrated using epidemic models with varying levels of complexity applied to data from the 1918 influenza pandemic in San Francisco. A tutorial video that demonstrates the functionality of the toolbox is included. 
    more » « less
  5. ; ; (, Royal Society Open Science)
    Mathematical models of neuronal networks play a crucial role in understanding sleep dynamics and associated disorders. However, validating these models through parameter estimation remains a significant challenge. In this work, we introduce an automated parameter estimation framework for sleep models that satisfy two key assumptions: (i) they consist of competing neuronal populations, each driving a distinct sleep stage (stage-promoting), and (ii) their dynamics evolve independently of weakly observed variables or external inputs (self-contained). We apply our method to a system of coupled nonlinear ordinary differential equations (ODEs) representing three interacting neuronal populations. Direct firing rates of these populations are typically unobservable, and hypnograms provide only the dominant sleep stage at each time point. Despite the limited information available in hypnograms, we successfully estimate ODE parameters for the underlying neuronal population model directly from hypnogram data. We use a smoothed winner-takes-all strategy within a constrained minimization framework, reformulate the problem in an unconstrained setting through the Lagrangian, and derive the corresponding optimality conditions from state and adjoint equations. A projected nonlinear conjugate gradient scheme is then used to estimate the parameters numerically. We validate our approach by accurately reconstructing 111 out of 139 hypnograms from the Sleep-EDF database. The inferred population-level parameters provide insights into sleep regulation by capturing interaction strengths, timescale constants and non-rapid eye movement-related variability. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email