Motivated by recent progress in data assimilation, we develop an algorithm to dynamically learn the parameters of a chaotic system from partial observations. Under reasonable assumptions, we supply a rigorous analytical proof that guarantees the convergence of this algorithm to the true parameter values when the system in question is the classic threedimensional Lorenz system. Such a result appears to be the first of its kind for dynamical parameter estimation of nonlinear systems. Computationally, we demonstrate the efficacy of this algorithm on the Lorenz system by recovering any proper subset of the three nondimensional parameters of the system, so long as a corresponding subset of the state is observable. We moreover probe the limitations of the algorithm by identifying dynamical regimes under which certain parameters cannot be effectively inferred having only observed certain state variables. In such cases, modifications to the algorithm are proposed that ultimately result in recovery of the parameter. Lastly, computational evidence is provided that supports the efficacy of the algorithm well beyond the hypotheses specified by the theorem, including in the presence of noisy observations, stochastic forcing, and the case where the observations are discrete and sparse in time.
more » « less NSFPAR ID:
 10343880
 Date Published:
 Journal Name:
 Discrete and Continuous Dynamical Systems
 Volume:
 42
 Issue:
 8
 ISSN:
 10780947
 Page Range / eLocation ID:
 3809
 Format(s):
 Medium: X
 Sponsoring Org:
 National Science Foundation
More Like this

null (Ed.)Recent advances in computing algorithms and hardware have rekindled interest in developing highaccuracy, lowcost surrogate models for simulating physical systems. The idea is to replace expensive numerical integration of complex coupled partial differential equations at fine time scales performed on supercomputers, with machinelearned surrogates that efficiently and accurately forecast future system states using data sampled from the underlying system. One particularly popular technique being explored within the weather and climate modelling community is the echo state network (ESN), an attractive alternative to other wellknown deep learning architectures. Using the classical Lorenz 63 system, and the three tier multiscale Lorenz 96 system (Thornes T, Duben P, Palmer T. 2017 Q. J. R. Meteorol. Soc. 143 , 897–908. ( doi:10.1002/qj.2974 )) as benchmarks, we realize that previously studied stateoftheart ESNs operate in two distinct regimes, corresponding to low and high spectral radius (LSR/HSR) for the sparse, randomly generated, reservoir recurrence matrix. Using knowledge of the mathematical structure of the Lorenz systems along with systematic ablation and hyperparameter sensitivity analyses, we show that stateoftheart LSRESNs reduce to a polynomial regression model which we call DomainDriven Regularized Regression (D2R2). Interestingly, D2R2 is a generalization of the wellknown SINDy algorithm (Brunton SL, Proctor JL, Kutz JN. 2016 Proc. Natl Acad. Sci. USA 113 , 3932–3937. ( doi:10.1073/pnas.1517384113 )). We also show experimentally that LSRESNs (Chattopadhyay A, Hassanzadeh P, Subramanian D. 2019 ( http://arxiv.org/abs/1906.08829 )) outperform HSR ESNs (Pathak J, Hunt B, Girvan M, Lu Z, Ott E. 2018 Phys. Rev. Lett. 120 , 024102. ( doi:10.1103/PhysRevLett.120.024102 )) while D2R2 dominates both approaches. A significant goal in constructing surrogates is to cope with barriers to scaling in weather prediction and simulation of dynamical systems that are imposed by time and energy consumption in supercomputers. Inexact computing has emerged as a novel approach to helping with scaling. In this paper, we evaluate the performance of three models (LSRESN, HSRESN and D2R2) by varying the precision or word size of the computation as our inexactnesscontrolling parameter. For precisions of 64, 32 and 16 bits, we show that, surprisingly, the least expensive D2R2 method yields the most robust results and the greatest savings compared to ESNs. Specifically, D2R2 achieves 68 × in computational savings, with an additional 2 × if precision reductions are also employed, outperforming ESN variants by a large margin. This article is part of the theme issue ‘Machine learning for weather and climate modelling’.more » « less

Abstract For data assimilation to provide faithful state estimates for dynamical models, specifications of observation uncertainty need to be as accurate as possible. Innovationbased methods based on Desroziers diagnostics, are commonly used to estimate observation uncertainty, but such methods can depend greatly on the prescribed background uncertainty. For ensemble data assimilation, this uncertainty comes from statistics calculated from ensemble forecasts, which require inflation and localization to address under sampling. In this work, we use an ensemble Kalman filter (EnKF) with a lowdimensional Lorenz model to investigate the interplay between the Desroziers method and inflation. Two inflation techniques are used for this purpose: 1) a rigorously tuned fixed multiplicative scheme and 2) an adaptive statespace scheme. We document how inaccuracies in observation uncertainty affect errors in EnKF posteriors and study the combined impacts of misspecified initial observation uncertainty, sampling error, and model error on Desroziers estimates. We find that whether observation uncertainty is over or underestimated greatly affects the stability of data assimilation and the accuracy of Desroziers estimates and that preference should be given to initial overestimates. Inline estimates of Desroziers tend to remove the dependence between ensemble spread–skill and the initially prescribed observation error. In addition, we find that the inclusion of model error introduces spurious correlations in observation uncertainty estimates. Further, we note that the adaptive inflation scheme is less robust than fixed inflation at mitigating multiple sources of error. Last, sampling error strongly exacerbates existing sources of error and greatly degrades EnKF estimates, which translates into biased Desroziers estimates of observation error covariance.
Significance Statement To generate accurate predictions of various components of the Earth system, numerical models require an accurate specification of state variables at our current time. This step adopts a probabilistic consideration of our current state estimate versus information provided from environmental measurements of the true state. Various strategies exist for estimating uncertainty in observations within this framework, but are sensitive to a host of assumptions, which are investigated in this study.

ISALT: Inferencebased schemes adaptive to large timestepping for locally Lipschitz ergodic systems
Efficient simulation of SDEs is essential in many applications, particularly for ergodic systems that demand efficient simulation of both shorttime dynamics and largetime statistics. However, locally Lipschitz SDEs often require special treatments such as implicit schemes with small timesteps to accurately simulate the ergodic measures. We introduce a framework to construct inferencebased schemes adaptive to large timesteps (ISALT) from data, achieving a reduction in time by several orders of magnitudes. The key is the statistical learning of an approximation to the infinitedimensional discretetime flow map. We explore the use of numerical schemes (such as the EulerMaruyama, the hybrid RK4, and an implicit scheme) to derive informed basis functions, leading to a parameter inference problem. We introduce a scalable algorithm to estimate the parameters by least squares, and we prove the convergence of the estimators as data size increases.
We test the ISALT on three nonglobally Lipschitz SDEs: the 1D doublewell potential, a 2D multiscale gradient system, and the 3D stochastic Lorenz equation with a degenerate noise. Numerical results show that ISALT can tolerate timestep magnitudes larger than plain numerical schemes. It reaches optimal accuracy in reproducing the invariant measure when the timestep is mediumlarge.

We study the problem of observation selection in a resourceconstrained networked sensing system, where the objective is to select a small subset of observations from a large network to perform a state estimation task. When the measurements are gathered using nonlinear systems, majority of prior work resort to approximation techniques such as linearization of the measurement model to utilize the methods developed for linear models, e.g., (weak) submodular objectives and greedy selection schemes. In contrast, when the measurement model is quadratic, e.g., the range measurements in a radar system, by exploiting a connection to the classical Van Trees' inequality, we derive new optimality criteria without distorting the relational structure of the measurement model. We further show that under certain conditions these optimality criteria are monotone and (weak) submodular set functions. These results enable us to develop an efficient greedy observation selection algorithm uniquely tailored for constrained networked sensing systems following quadratic models and provide theoretical bounds on its achievable utility. Extensive numerical experiments demonstrate efficacy of the proposed framework.more » « less

Abstract Our recent work on linear and affine dynamical systems has laid out a general framework for inferring the parameters of a differential equation model from a discrete set of data points collected from a system being modeled. It introduced a new class of inverse problems where qualitative information about the parameters and the associated dynamics of the system is determined for regions of the data space, rather than just for isolated experiments. Rigorous mathematical results have justified this approach and have identified common features that arise for certain classes of integrable models. In this work we present a thorough numerical investigation that shows that several of these core features extend to a paradigmatic linearinparameters model, the Lotka–Volterra (LV) system, which we consider in the conservative case as well as under the addition of terms that perturb the system away from this regime. A central construct for this analysis is a concise representation of parameter and dynamical features in the data space that we call the
P _{n}diagram, which is particularly useful for visualization of the qualitative dependence of the system dynamics on data for lowdimensional (smalln ) systems. Our work also exposes some new properties related to nonuniqueness that arise for these LV systems, with nonuniqueness manifesting as a multilayered structure in the associatedP _{2}diagrams.