skip to main content

Title: Dynamically learning the parameters of a chaotic system using partial observations

Motivated by recent progress in data assimilation, we develop an algorithm to dynamically learn the parameters of a chaotic system from partial observations. Under reasonable assumptions, we supply a rigorous analytical proof that guarantees the convergence of this algorithm to the true parameter values when the system in question is the classic three-dimensional Lorenz system. Such a result appears to be the first of its kind for dynamical parameter estimation of nonlinear systems. Computationally, we demonstrate the efficacy of this algorithm on the Lorenz system by recovering any proper subset of the three non-dimensional parameters of the system, so long as a corresponding subset of the state is observable. We moreover probe the limitations of the algorithm by identifying dynamical regimes under which certain parameters cannot be effectively inferred having only observed certain state variables. In such cases, modifications to the algorithm are proposed that ultimately result in recovery of the parameter. Lastly, computational evidence is provided that supports the efficacy of the algorithm well beyond the hypotheses specified by the theorem, including in the presence of noisy observations, stochastic forcing, and the case where the observations are discrete and sparse in time.

more » « less
Award ID(s):
1953346 2206762
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
Discrete and Continuous Dynamical Systems
Page Range / eLocation ID:
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    Recent advances in computing algorithms and hardware have rekindled interest in developing high-accuracy, low-cost surrogate models for simulating physical systems. The idea is to replace expensive numerical integration of complex coupled partial differential equations at fine time scales performed on supercomputers, with machine-learned surrogates that efficiently and accurately forecast future system states using data sampled from the underlying system. One particularly popular technique being explored within the weather and climate modelling community is the echo state network (ESN), an attractive alternative to other well-known deep learning architectures. Using the classical Lorenz 63 system, and the three tier multi-scale Lorenz 96 system (Thornes T, Duben P, Palmer T. 2017 Q. J. R. Meteorol. Soc. 143 , 897–908. ( doi:10.1002/qj.2974 )) as benchmarks, we realize that previously studied state-of-the-art ESNs operate in two distinct regimes, corresponding to low and high spectral radius (LSR/HSR) for the sparse, randomly generated, reservoir recurrence matrix. Using knowledge of the mathematical structure of the Lorenz systems along with systematic ablation and hyperparameter sensitivity analyses, we show that state-of-the-art LSR-ESNs reduce to a polynomial regression model which we call Domain-Driven Regularized Regression (D2R2). Interestingly, D2R2 is a generalization of the well-known SINDy algorithm (Brunton SL, Proctor JL, Kutz JN. 2016 Proc. Natl Acad. Sci. USA 113 , 3932–3937. ( doi:10.1073/pnas.1517384113 )). We also show experimentally that LSR-ESNs (Chattopadhyay A, Hassanzadeh P, Subramanian D. 2019 ( )) outperform HSR ESNs (Pathak J, Hunt B, Girvan M, Lu Z, Ott E. 2018 Phys. Rev. Lett. 120 , 024102. ( doi:10.1103/PhysRevLett.120.024102 )) while D2R2 dominates both approaches. A significant goal in constructing surrogates is to cope with barriers to scaling in weather prediction and simulation of dynamical systems that are imposed by time and energy consumption in supercomputers. Inexact computing has emerged as a novel approach to helping with scaling. In this paper, we evaluate the performance of three models (LSR-ESN, HSR-ESN and D2R2) by varying the precision or word size of the computation as our inexactness-controlling parameter. For precisions of 64, 32 and 16 bits, we show that, surprisingly, the least expensive D2R2 method yields the most robust results and the greatest savings compared to ESNs. Specifically, D2R2 achieves 68 × in computational savings, with an additional 2 × if precision reductions are also employed, outperforming ESN variants by a large margin. This article is part of the theme issue ‘Machine learning for weather and climate modelling’. 
    more » « less
  2. Abstract

    For data assimilation to provide faithful state estimates for dynamical models, specifications of observation uncertainty need to be as accurate as possible. Innovation-based methods based on Desroziers diagnostics, are commonly used to estimate observation uncertainty, but such methods can depend greatly on the prescribed background uncertainty. For ensemble data assimilation, this uncertainty comes from statistics calculated from ensemble forecasts, which require inflation and localization to address under sampling. In this work, we use an ensemble Kalman filter (EnKF) with a low-dimensional Lorenz model to investigate the interplay between the Desroziers method and inflation. Two inflation techniques are used for this purpose: 1) a rigorously tuned fixed multiplicative scheme and 2) an adaptive state-space scheme. We document how inaccuracies in observation uncertainty affect errors in EnKF posteriors and study the combined impacts of misspecified initial observation uncertainty, sampling error, and model error on Desroziers estimates. We find that whether observation uncertainty is over- or underestimated greatly affects the stability of data assimilation and the accuracy of Desroziers estimates and that preference should be given to initial overestimates. Inline estimates of Desroziers tend to remove the dependence between ensemble spread–skill and the initially prescribed observation error. In addition, we find that the inclusion of model error introduces spurious correlations in observation uncertainty estimates. Further, we note that the adaptive inflation scheme is less robust than fixed inflation at mitigating multiple sources of error. Last, sampling error strongly exacerbates existing sources of error and greatly degrades EnKF estimates, which translates into biased Desroziers estimates of observation error covariance.

    Significance Statement

    To generate accurate predictions of various components of the Earth system, numerical models require an accurate specification of state variables at our current time. This step adopts a probabilistic consideration of our current state estimate versus information provided from environmental measurements of the true state. Various strategies exist for estimating uncertainty in observations within this framework, but are sensitive to a host of assumptions, which are investigated in this study.

    more » « less
  3. Efficient simulation of SDEs is essential in many applications, particularly for ergodic systems that demand efficient simulation of both short-time dynamics and large-time statistics. However, locally Lipschitz SDEs often require special treatments such as implicit schemes with small time-steps to accurately simulate the ergodic measures. We introduce a framework to construct inference-based schemes adaptive to large time-steps (ISALT) from data, achieving a reduction in time by several orders of magnitudes. The key is the statistical learning of an approximation to the infinite-dimensional discrete-time flow map. We explore the use of numerical schemes (such as the Euler-Maruyama, the hybrid RK4, and an implicit scheme) to derive informed basis functions, leading to a parameter inference problem. We introduce a scalable algorithm to estimate the parameters by least squares, and we prove the convergence of the estimators as data size increases.

    We test the ISALT on three non-globally Lipschitz SDEs: the 1D double-well potential, a 2D multiscale gradient system, and the 3D stochastic Lorenz equation with a degenerate noise. Numerical results show that ISALT can tolerate time-step magnitudes larger than plain numerical schemes. It reaches optimal accuracy in reproducing the invariant measure when the time-step is medium-large.

    more » « less
  4. We study the problem of observation selection in a resource-constrained networked sensing system, where the objective is to select a small subset of observations from a large network to perform a state estimation task. When the measurements are gathered using nonlinear systems, majority of prior work resort to approximation techniques such as linearization of the measurement model to utilize the methods developed for linear models, e.g., (weak) submodular objectives and greedy selection schemes. In contrast, when the measurement model is quadratic, e.g., the range measurements in a radar system, by exploiting a connection to the classical Van Trees' inequality, we derive new optimality criteria without distorting the relational structure of the measurement model. We further show that under certain conditions these optimality criteria are monotone and (weak) submodular set functions. These results enable us to develop an efficient greedy observation selection algorithm uniquely tailored for constrained networked sensing systems following quadratic models and provide theoretical bounds on its achievable utility. Extensive numerical experiments demonstrate efficacy of the proposed framework. 
    more » « less
  5. Abstract

    Our recent work on linear and affine dynamical systems has laid out a general framework for inferring the parameters of a differential equation model from a discrete set of data points collected from a system being modeled. It introduced a new class of inverse problems where qualitative information about the parameters and the associated dynamics of the system is determined for regions of the data space, rather than just for isolated experiments. Rigorous mathematical results have justified this approach and have identified common features that arise for certain classes of integrable models. In this work we present a thorough numerical investigation that shows that several of these core features extend to a paradigmatic linear-in-parameters model, the Lotka–Volterra (LV) system, which we consider in the conservative case as well as under the addition of terms that perturb the system away from this regime. A central construct for this analysis is a concise representation of parameter and dynamical features in the data space that we call thePn-diagram, which is particularly useful for visualization of the qualitative dependence of the system dynamics on data for low-dimensional (smalln) systems. Our work also exposes some new properties related to non-uniqueness that arise for these LV systems, with non-uniqueness manifesting as a multi-layered structure in the associatedP2-diagrams.

    more » « less