More Like this

1. A dynamical Gaussian, lognormal, and reverse lognormal Kalman filter

   https://doi.org/10.1002/qj.4595  

   Van Loon, Senne; Fletcher, Steven J. (November 2023, Quarterly Journal of the Royal Meteorological Society)

   Abstract We derive a generalization of the Kalman filter that allows for non‐Gaussian background and observation errors. The Gaussian assumption is replaced by considering that the errors come from a mixed distribution of Gaussian, lognormal, and reverse lognormal random variables. We detail the derivation for reverse lognormal errors and extend the results to mixed distributions, where the number of Gaussian, lognormal, and reverse lognormal state variables can change dynamically every analysis time. We test the dynamical mixed Kalman filter robustly on two different systems based on the Lorenz 1963 model, and demonstrate that non‐Gaussian techniques generally improve the analysis skill if the observations are sparse and uncertain, compared with the Gaussian Kalman filter. 

   more » « less
2. Score matching filters for Gaussian Markov random fields with a linear model of the precision matrix

   https://doi.org/10.3934/fods.2021030  

   Turčičová, Marie; Mandel, Jan; Eben, Kryštof (January 2021, Foundations of Data Science)

   We present an ensemble filtering method based on a linear model for the precision matrix (the inverse of the covariance) with the parameters determined by Score Matching Estimation. The method provides a rigorous covariance regularization when the underlying random field is Gaussian Markov. The parameters are found by solving a system of linear equations. The analysis step uses the inverse formulation of the Kalman update. Several filter versions, differing in the construction of the analysis ensemble, are proposed, as well as a Score matching version of the Extended Kalman Filter. 

   more » « less
3. Dynamical Gaussian, lognormal, and reverse lognormal maximum‐likelihood ensemble filter

   https://doi.org/10.1002/qj.4706  

   Van_Loon, Senne; Fletcher, Steven J; Zupanski, Milija (April 2024, Quarterly Journal of the Royal Meteorological Society)

   We present a non‐Gaussian ensemble data assimilation method based on the maximum‐likelihood ensemble filter, which allows for any combination of Gaussian, lognormal, and reverse lognormal errors in both the background and the observations. The technique is fully nonlinear, does not require a tangent linear model, and uses a Hessian preconditioner to minimise the cost function efficiently in ensemble space. When the Gaussian assumption is relaxed, the results show significant improvements in the analysis skill within two atmospheric toy models, and the performance of data assimilation systems for (semi)bounded variables is expected to improve. 

   more » « less
4. A hybrid particle-ensemble Kalman filter for problems with medium nonlinearity

   https://doi.org/10.1371/journal.pone.0248266  

   Grooms, Ian; Robinson, Gregor (March 2021, PLOS ONE)

   Hoteit, Ibrahim (Ed.)

   A hybrid particle ensemble Kalman filter is developed for problems with medium non-Gaussianity, i.e. problems where the prior is very non-Gaussian but the posterior is approximately Gaussian. Such situations arise, e.g., when nonlinear dynamics produce a non-Gaussian forecast but a tight Gaussian likelihood leads to a nearly-Gaussian posterior. The hybrid filter starts by factoring the likelihood. First the particle filter assimilates the observations with one factor of the likelihood to produce an intermediate prior that is close to Gaussian, and then the ensemble Kalman filter completes the assimilation with the remaining factor. How the likelihood gets split between the two stages is determined in such a way to ensure that the particle filter avoids collapse, and particle degeneracy is broken by a mean-preserving random orthogonal transformation. The hybrid is tested in a simple two-dimensional (2D) problem and a multiscale system of ODEs motivated by the Lorenz-‘96 model. In the 2D problem it outperforms both a pure particle filter and a pure ensemble Kalman filter, and in the multiscale Lorenz-‘96 model it is shown to outperform a pure ensemble Kalman filter, provided that the ensemble size is large enough. 

   more » « less
5. Detection of Gaussian Attacks in Power Systems under a Scalable Kalman Consensus Filter Framework

   Fernandes, Arnold; Bo, Rui; Kimball, Jonathan; McMillin, Bruce (July 2021, 2021 IEEE PES General Meeting)

   null (Ed.)

   The dynamic non-linear state-space model of a power-system consisting of synchronous generators, buses, and static loads has been linearized and a linear measurement function has been considered. A distributed dynamic framework for estimating the state vector of the power system has been designed here. This framework employs a type of distributed Kalman filter (DKF) known as a Kalman consensus filter (KCF) which is located at distributed control centers (DCCs) that fuse locally available noise ridden measurements, state vector estimates of neighboring control centers, and a prediction obtained by the linearized model to obtain a filtered state vector estimate. Further, the local residual at each control center is checked by a median chi-squared detector designed here for bad data/Gaussian attack detection. Simulation results show the working of the KCF for an 8 bus 5 generator system, and the efficacy of the median chi-squared detector in detecting the DCC affected by Gaussian attacks. 

   more » « less