- Award ID(s):
- 1944678
- NSF-PAR ID:
- 10334686
- Date Published:
- Journal Name:
- Industrial engineering chemistry research
- Volume:
- 60
- Issue:
- 45
- ISSN:
- 1520-5045
- Page Range / eLocation ID:
- 16330–16344
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE). This observation motivated the introduction of so-called Neural ODEs, which allow more general discretization schemes with adaptive time stepping. Here, we propose ANODEV2, which is an extension of this approach that allows evolution of the neural network parameters, in a coupled ODE-based formulation. The Neural ODE method introduced earlier is in fact a special case of this new framework. We present the formulation of ANODEV2, derive optimality conditions, and implement the coupled framework in PyTorch. We present empirical results using several different configurations of ANODEV2, testing them on multiple models on CIFAR-10. We report results showing that this coupled ODE-based framework is indeed trainable, and that it achieves higher accuracy, as compared to the baseline models as well as the recently-proposed Neural ODE approach.more » « less
-
Abstract Many scientific problems focus on observed patterns of change or on how to design a system to achieve particular dynamics. Those problems often require fitting differential equation models to target trajectories. Fitting such models can be difficult because each evaluation of the fit must calculate the distance between the model and target patterns at numerous points along a trajectory. The gradient of the fit with respect to the model parameters can be challenging to compute. Recent technical advances in automatic differentiation through numerical differential equation solvers potentially change the fitting process into a relatively easy problem, opening up new possibilities to study dynamics. However, application of the new tools to real data may fail to achieve a good fit. This article illustrates how to overcome a variety of common challenges, using the classic ecological data for oscillations in hare and lynx populations. Models include simple ordinary differential equations (ODEs) and neural ordinary differential equations (NODEs), which use artificial neural networks to estimate the derivatives of differential equation systems. Comparing the fits obtained with ODEs versus NODEs, representing small and large parameter spaces, and changing the number of variable dimensions provide insight into the geometry of the observed and model trajectories. To analyze the quality of the models for predicting future observations, a Bayesian‐inspired preconditioned stochastic gradient Langevin dynamics (pSGLD) calculation of the posterior distribution of predicted model trajectories clarifies the tendency for various models to underfit or overfit the data. Coupling fitted differential equation systems with pSGLD sampling provides a powerful way to study the properties of optimization surfaces, raising an analogy with mutation‐selection dynamics on fitness landscapes.
-
Purpose: Little quantitative or mechanistic information about tear film breakup can be determined directly via current imaging techniques. In this paper, we present simplified mathematical models based on two proposed mechanisms of tear film breakup: evaporation of water from the tear film and tangential fluid flow within the tear film. We use our models to determine whether one or a combination of the two mechanisms causes tear film breakup in a variety of instances. In this study, we estimate related breakup parameters that cannot currently be measured in breakup during subject trials, such as tear film osmolarity and thinning rates. The present study validates our procedure against previous work.Methods: Five ordinary differential equation models for tear film thinning were designed that model evaporation, osmosis, and various types of tangential flow. Eight tear film breakup instances occurring within a time interval of 1–8 s postblink of five healthy subjects thatwere identified in fluorescence images in previous work were fit with these five models. The fitting procedure used a nonlinear least squares optimization that minimized the difference of the computed theoretical fluorescent intensity from the models and the experimental fluorescent intensity from the images. The optimization was conducted over the evaporation rate and up to three tangential flow rate parameters. The smallest norm of the difference was determined to correspond to the model that best explained the tear film dynamics.Results: All of the breakup instances were best fit by models with time-dependent tangential flow. Our optimal parameter values and thinning rate as well as tangential fluid flow profiles compare well with previous partial differential equation model results in most instances.Conclusion: Our fitting results suggest that a combination of tangential fluid flow and evaporation cause most of the breakup instances. Comparison with results from previous work suggests that the simplified models can capture the essential tear film dynamics in most cases, thereby validating this procedure for wider usage.more » « less
-
We investigate approximate Bayesian inference techniques for nonlinear systems described by ordinary differential equation (ODE) models. In particular, the approximations will be based on set-valued reachability analysis approaches, yielding approximate models for the posterior distribution. Nonlinear ODEs are widely used to mathematically describe physical and biological models. However, these models are often described by parameters that are not directly measurable and have an impact on the system behaviors. Often, noisy measurement data combined with physical/biological intuition serve as the means for finding appropriate values of these parameters.Our approach operates under a Bayesian framework, given prior distribution over the parameter space and noisy observations under a known sampling distribution. We explore subsets of the space of model parameters, computing bounds on the likelihood for each subset. This is performed using nonlinear set-valued reachability analysis that is made faster by means of linearization around a reference trajectory. The tiling of the parameter space can be adaptively refined to make bounds on the likelihood tighter. We evaluate our approach on a variety of nonlinear benchmarks and compare our results with Markov Chain Monte Carlo and Sequential Monte Carlo approaches.
-
Mathematical models based on systems of ordinary differential equations (ODEs) are frequently applied in various scientific fields to assess hypotheses, estimate key model parameters, and generate predictions about the system's state. To support their application, we present a comprehensive, easy‐to‐use, and flexible MATLAB toolbox,
QuantDiffForecast , and associated tutorial to estimate parameters and generate short‐term forecasts with quantified uncertainty from dynamical models based on systems of ODEs. We provide software (https://github.com/gchowell/paramEstimation_forecasting_ODEmodels/ ) and detailed guidance on estimating parameters and forecasting time‐series trajectories that are characterized using ODEs with quantified uncertainty through a parametric bootstrapping approach. It includes functions that allow the user to infer model parameters and assess forecasting performance for different ODE models specified by the user, using different estimation methods and error structures in the data. The tutorial is intended for a diverse audience, including students training in dynamic systems, and will be broadly applicable to estimate parameters and generate forecasts from models based on ODEs. The functions included in the toolbox are illustrated using epidemic models with varying levels of complexity applied to data from the 1918 influenza pandemic in San Francisco. A tutorial video that demonstrates the functionality of the toolbox is included.