- Award ID(s):
- 1944678
- NSF-PAR ID:
- 10334686
- Date Published:
- Journal Name:
- Industrial engineering chemistry research
- Volume:
- 60
- Issue:
- 45
- ISSN:
- 1520-5045
- Page Range / eLocation ID:
- 16330–16344
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE). This observation motivated the introduction of so-called Neural ODEs, which allow more general discretization schemes with adaptive time stepping. Here, we propose ANODEV2, which is an extension of this approach that allows evolution of the neural network parameters, in a coupled ODE-based formulation. The Neural ODE method introduced earlier is in fact a special case of this new framework. We present the formulation of ANODEV2, derive optimality conditions, and implement the coupled framework in PyTorch. We present empirical results using several different configurations of ANODEV2, testing them on multiple models on CIFAR-10. We report results showing that this coupled ODE-based framework is indeed trainable, and that it achieves higher accuracy, as compared to the baseline models as well as the recently-proposed Neural ODE approach.more » « less
-
Abstract Many scientific problems focus on observed patterns of change or on how to design a system to achieve particular dynamics. Those problems often require fitting differential equation models to target trajectories. Fitting such models can be difficult because each evaluation of the fit must calculate the distance between the model and target patterns at numerous points along a trajectory. The gradient of the fit with respect to the model parameters can be challenging to compute. Recent technical advances in automatic differentiation through numerical differential equation solvers potentially change the fitting process into a relatively easy problem, opening up new possibilities to study dynamics. However, application of the new tools to real data may fail to achieve a good fit. This article illustrates how to overcome a variety of common challenges, using the classic ecological data for oscillations in hare and lynx populations. Models include simple ordinary differential equations (ODEs) and neural ordinary differential equations (NODEs), which use artificial neural networks to estimate the derivatives of differential equation systems. Comparing the fits obtained with ODEs versus NODEs, representing small and large parameter spaces, and changing the number of variable dimensions provide insight into the geometry of the observed and model trajectories. To analyze the quality of the models for predicting future observations, a Bayesian‐inspired preconditioned stochastic gradient Langevin dynamics (pSGLD) calculation of the posterior distribution of predicted model trajectories clarifies the tendency for various models to underfit or overfit the data. Coupling fitted differential equation systems with pSGLD sampling provides a powerful way to study the properties of optimization surfaces, raising an analogy with mutation‐selection dynamics on fitness landscapes.
-
Purpose: Little quantitative or mechanistic information about tear film breakup can be determined directly via current imaging techniques. In this paper, we present simplified mathematical models based on two proposed mechanisms of tear film breakup: evaporation of water from the tear film and tangential fluid flow within the tear film. We use our models to determine whether one or a combination of the two mechanisms causes tear film breakup in a variety of instances. In this study, we estimate related breakup parameters that cannot currently be measured in breakup during subject trials, such as tear film osmolarity and thinning rates. The present study validates our procedure against previous work.Methods: Five ordinary differential equation models for tear film thinning were designed that model evaporation, osmosis, and various types of tangential flow. Eight tear film breakup instances occurring within a time interval of 1–8 s postblink of five healthy subjects thatwere identified in fluorescence images in previous work were fit with these five models. The fitting procedure used a nonlinear least squares optimization that minimized the difference of the computed theoretical fluorescent intensity from the models and the experimental fluorescent intensity from the images. The optimization was conducted over the evaporation rate and up to three tangential flow rate parameters. The smallest norm of the difference was determined to correspond to the model that best explained the tear film dynamics.Results: All of the breakup instances were best fit by models with time-dependent tangential flow. Our optimal parameter values and thinning rate as well as tangential fluid flow profiles compare well with previous partial differential equation model results in most instances.Conclusion: Our fitting results suggest that a combination of tangential fluid flow and evaporation cause most of the breakup instances. Comparison with results from previous work suggests that the simplified models can capture the essential tear film dynamics in most cases, thereby validating this procedure for wider usage.more » « less
-
We investigate approximate Bayesian inference techniques for nonlinear systems described by ordinary differential equation (ODE) models. In particular, the approximations will be based on set-valued reachability analysis approaches, yielding approximate models for the posterior distribution. Nonlinear ODEs are widely used to mathematically describe physical and biological models. However, these models are often described by parameters that are not directly measurable and have an impact on the system behaviors. Often, noisy measurement data combined with physical/biological intuition serve as the means for finding appropriate values of these parameters.Our approach operates under a Bayesian framework, given prior distribution over the parameter space and noisy observations under a known sampling distribution. We explore subsets of the space of model parameters, computing bounds on the likelihood for each subset. This is performed using nonlinear set-valued reachability analysis that is made faster by means of linearization around a reference trajectory. The tiling of the parameter space can be adaptively refined to make bounds on the likelihood tighter. We evaluate our approach on a variety of nonlinear benchmarks and compare our results with Markov Chain Monte Carlo and Sequential Monte Carlo approaches.
-
Abstract We introduce the Weak-form Estimation of Nonlinear Dynamics (WENDy) method for estimating model parameters for non-linear systems of ODEs. Without relying on any numerical differential equation solvers, WENDy computes accurate estimates and is robust to large (biologically relevant) levels of measurement noise. For low dimensional systems with modest amounts of data, WENDy is competitive with conventional forward solver-based nonlinear least squares methods in terms of speed and accuracy. For both higher dimensional systems and stiff systems, WENDy is typically both faster (often by orders of magnitude) and more accurate than forward solver-based approaches. The core mathematical idea involves an efficient conversion of the strong form representation of a model to its weak form, and then solving a regression problem to perform parameter inference. The core statistical idea rests on the Errors-In-Variables framework, which necessitates the use of the iteratively reweighted least squares algorithm. Further improvements are obtained by using orthonormal test functions, created from a set of
bump functions of varying support sizes.We demonstrate the high robustness and computational efficiency by applying WENDy to estimate parameters in some common models from population biology, neuroscience, and biochemistry, including logistic growth, Lotka-Volterra, FitzHugh-Nagumo, Hindmarsh-Rose, and a Protein Transduction Benchmark model. Software and code for reproducing the examples is available at$$C^{\infty }$$ https://github.com/MathBioCU/WENDy .