More Like this

1. A dynamical Gaussian, lognormal, and reverse lognormal Kalman filter

   https://doi.org/10.1002/qj.4595  

   Van Loon, Senne; Fletcher, Steven J. (November 2023, Quarterly Journal of the Royal Meteorological Society)

   Abstract We derive a generalization of the Kalman filter that allows for non‐Gaussian background and observation errors. The Gaussian assumption is replaced by considering that the errors come from a mixed distribution of Gaussian, lognormal, and reverse lognormal random variables. We detail the derivation for reverse lognormal errors and extend the results to mixed distributions, where the number of Gaussian, lognormal, and reverse lognormal state variables can change dynamically every analysis time. We test the dynamical mixed Kalman filter robustly on two different systems based on the Lorenz 1963 model, and demonstrate that non‐Gaussian techniques generally improve the analysis skill if the observations are sparse and uncertain, compared with the Gaussian Kalman filter. 

   more » « less
2. Lognormal and Mixed Gaussian–Lognormal Kalman Filters

   https://doi.org/10.1175/MWR-D-22-0072.1  

   Fletcher, Steven J.; Zupanski, Milija; Goodliff, Michael R.; Kliewer, Anton J.; Jones, Andrew S.; Forsythe, John M.; Wu, Ting-Chi; Hossen, Md. Jakir; Van Loon, Senne (March 2023, Monthly Weather Review)

   Abstract In this paper we present the derivation of two new forms of the Kalman filter equations; the first is for a pure lognormally distributed random variable, while the second set of Kalman filter equations will be for a combination of Gaussian and lognormally distributed random variables. We show that the appearance is similar to that of the Gaussian-based equations, but that the analysis state is a multivariate median and not the mean. We also show results of the mixed distribution Kalman filter with the Lorenz 1963 model with lognormal errors for the background and observations of the z component, and compare them to analysis results from a traditional Gaussian-based extended Kalman filter and show that under certain circumstances the new approach produces more accurate results. 

   more » « less
3. A Quantile-Conserving Ensemble Filter Based on Kernel-Density Estimation

   https://doi.org/10.3390/rs16132377  

   Grooms, Ian; Riedel, Christopher (July 2024, Remote Sensing)

   Ensemble Kalman filters are an efficient class of algorithms for large-scale ensemble data assimilation, but their performance is limited by their underlying Gaussian approximation. A two-step framework for ensemble data assimilation allows this approximation to be relaxed: The first step updates the ensemble in observation space, while the second step regresses the observation state update back to the state variables. This paper develops a new quantile-conserving ensemble filter based on kernel-density estimation and quadrature for the scalar first step of the two-step framework. It is shown to perform well in idealized non-Gaussian problems, as well as in an idealized model of assimilating observations of sea-ice concentration. 

   more » « less
4. Ensemble Riemannian data assimilation over the Wasserstein space

   https://doi.org/10.5194/npg-28-295-2021  

   Tamang, Sagar K.; Ebtehaj, Ardeshir; van Leeuwen, Peter J.; Zou, Dongmian; Lerman, Gilad (January 2021, Nonlinear Processes in Geophysics)

   null (Ed.)

   Abstract. In this paper, we present an ensemble data assimilation paradigm over a Riemannian manifold equipped with the Wasserstein metric. Unlike the Euclidean distance used in classic data assimilation methodologies, the Wasserstein metric can capture the translation and difference between the shapes of square-integrable probability distributions of the background state and observations. This enables us to formally penalize geophysical biases in state space with non-Gaussian distributions. The new approach is applied to dissipative and chaotic evolutionary dynamics, and its potential advantages and limitations are highlighted compared to the classic ensemble data assimilation approaches under systematic errors. 

   more » « less
5. Parameterizing Lognormal state space models using moment matching

   https://doi.org/10.1007/s10651-023-00570-x  

   Smith, John_W; Thomas, R_Quinn; Johnson, Leah_R (July 2023, Environmental and Ecological Statistics)

   Abstract In ecology, it is common for processes to be bounded based on physical constraints of the system. One common example is the positivity constraint, which applies to phenomena such as duration times, population sizes, and total stock of a system’s commodity. In this paper, we propose a novel method for parameterizing Lognormal state space models using an approach based on moment matching. Our method enforces the positivity constraint, allows for arbitrary mean evolution and variance structure, and has a closed-form Markov transition density which allows for more flexibility in fitting techniques. We discuss two existing Lognormal state space models and examine how they differ from the method presented here. We use 180 synthetic datasets to compare the forecasting performance under model misspecification and assess the estimation of precision parameters between our method and existing methods. We find that our models perform well under misspecification, and that fixing the observation variance both helps to improve estimation of the process variance and improves forecast performance. To test our method on a difficult problem, we compare the predictive performance of two Lognormal state space models in predicting the Leaf Area Index over a 151 day horizon by using a process-based ecosystem model to describe the temporal dynamics. We find that our moment matching model performs better than its competitor, and is better suited for intermediate predictive horizons. Overall, our study helps to inform practitioners about the importance of incorporating sensible dynamics when using models of complex systems to predict out-of-sample. 

   more » « less