Abstract Particle filters avoid parametric estimates for Bayesian posterior densities, which alleviates Gaussian assumptions in nonlinear regimes. These methods, however, are more sensitive to sampling errors than Gaussian-based techniques such as ensemble Kalman filters. A recent study by the authors introduced an iterative strategy for particle filters that match posterior moments—where iterations improve the filter’s ability to draw samples from non-Gaussian posterior densities. The iterations follow from a factorization of particle weights, providing a natural framework for combining particle filters with alternative filters to mitigate the impact of sampling errors. The current study introduces a novel approach to forming an adaptive hybrid data assimilation methodology, exploiting the theoretical strengths of nonparametric and parametric filters. At each data assimilation cycle, the iterative particle filter performs a sequence of updates while the prior sample distribution is non-Gaussian, then an ensemble Kalman filter provides the final adjustment when Gaussian distributions for marginal quantities are detected. The method employs the Shapiro–Wilk test to determine when to make the transition between filter algorithms, which has outstanding power for detecting departures from normality. Experiments using low-dimensional models demonstrate that the approach has a significant value, especially for nonhomogeneous observation networks and unknown model process errors. Moreover,more »
Novel Data Assimilation Algorithm for Nearshore Bathymetry
It can be expensive and difficult to collect direct bathymetry data for nearshore regions, especially in high-energy locations where there are temporally and spatially varying bathymetric features like sandbars. As a result, there has been increasing interest in remote assessment techniques for estimating bathymetry. Recent efforts have combined Kalman filter–based techniques with indirect video-based observations for bathymetry inversion. Here, we estimate nearshore bathymetry by utilizing observed wave celerity and wave height, which are related to bathymetry through phase-averaged wave dynamics. We present a modified compressed-state Kalman filter (CSKF) method, a fast and scalable Kalman filter method for linear and nonlinear problems with large numbers of unknowns and measurements, and apply it to two nearshore bathymetry estimation problems. To illustrate the robustness and accuracy of our method, we compare its performance with that of two ensemble-based approaches on twin bathymetry estimation problems with profiles based on surveys taken by the U.S. Army Corps of Engineer Field Research Facility (FRF) in Duck, North Carolina. We first consider an estimation problem for a temporally constant bathymetry profile. Then we estimate bathymetry as it evolves in time. Our results indicate that the CSKF method is more accurate and robust than the ensemble-based methods with more »
- Publication Date:
- NSF-PAR ID:
- 10125637
- Journal Name:
- Journal of Atmospheric and Oceanic Technology
- Volume:
- 36
- Issue:
- 4
- Page Range or eLocation-ID:
- p. 699-715
- ISSN:
- 0739-0572
- Publisher:
- American Meteorological Society
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Utilizing millimeter-wave (mmWave) frequencies for wireless communication in mobile systems is challenging since it requires continuous tracking of the beam direction. Recently, beam tracking techniques based on channel sparsity and/or Kalman filter-based techniques were proposed where the solutions use assumptions regarding the environment and device mobility that may not hold in practical scenarios. In this paper, we explore a machine learning-based approach to track the angle of arrival (AoA) for specific paths in realistic scenarios. In particular, we use a recurrent neural network (R-NN) structure with a modified cost function to track the AoA. We propose methods to train the network in sequential data, and study the performance of our proposed solution in comparison to an extended Kalman filter based solution in a realistic mmWave scenario based on stochastic channel model from the QuaDRiGa framework. Results show that our proposed solution outperforms an extended Kalman filter-based method by reducing the AoA outage probability, and thus reducing the need for frequent beam search.
-
Abstract The ensemble Kalman filter (EnKF) is a popular technique for data assimilation in high-dimensional nonlinear state-space models. The EnKF represents distributions of interest by an ensemble, which is a form of dimension reduction that enables straightforward forecasting even for complicated and expensive evolution operators. However, the EnKF update step involves estimation of the forecast covariance matrix based on the (often small) ensemble, which requires regularization. Many existing regularization techniques rely on spatial localization, which may ignore long-range dependence. Instead, our proposed approach assumes a sparse Cholesky factor of the inverse covariance matrix, and the nonzero Cholesky entries are further regularized. The resulting method is highly flexible and computationally scalable. In our numerical experiments, our approach was more accurate and less sensitive to misspecification of tuning parameters than tapering-based localization.
-
Abstract We consider Bayesian inference for large-scale inverse problems, where computational challenges arise from the need for repeated evaluations of an expensive forward model. This renders most Markov chain Monte Carlo approaches infeasible, since they typically require O ( 1 0 4 ) model runs, or more. Moreover, the forward model is often given as a black box or is impractical to differentiate. Therefore derivative-free algorithms are highly desirable. We propose a framework, which is built on Kalman methodology, to efficiently perform Bayesian inference in such inverse problems. The basic method is based on an approximation of the filtering distribution of a novel mean-field dynamical system, into which the inverse problem is embedded as an observation operator. Theoretical properties are established for linear inverse problems, demonstrating that the desired Bayesian posterior is given by the steady state of the law of the filtering distribution of the mean-field dynamical system, and proving exponential convergence to it. This suggests that, for nonlinear problems which are close to Gaussian, sequentially computing this law provides the basis for efficient iterative methods to approximate the Bayesian posterior. Ensemble methods are applied to obtain interacting particle system approximations of the filtering distribution of the mean-field model;more »
-
Score matching filters for Gaussian Markov random fields with a linear model of the precision matrix
We present an ensemble filtering method based on a linear model for the precision matrix (the inverse of the covariance) with the parameters determined by Score Matching Estimation. The method provides a rigorous covariance regularization when the underlying random field is Gaussian Markov. The parameters are found by solving a system of linear equations. The analysis step uses the inverse formulation of the Kalman update. Several filter versions, differing in the construction of the analysis ensemble, are proposed, as well as a Score matching version of the Extended Kalman Filter.
