skip to main content

Title: Challenges for Inline Observation Error Estimation in the Presence of Misspecified Background Uncertainty

For data assimilation to provide faithful state estimates for dynamical models, specifications of observation uncertainty need to be as accurate as possible. Innovation-based methods based on Desroziers diagnostics, are commonly used to estimate observation uncertainty, but such methods can depend greatly on the prescribed background uncertainty. For ensemble data assimilation, this uncertainty comes from statistics calculated from ensemble forecasts, which require inflation and localization to address under sampling. In this work, we use an ensemble Kalman filter (EnKF) with a low-dimensional Lorenz model to investigate the interplay between the Desroziers method and inflation. Two inflation techniques are used for this purpose: 1) a rigorously tuned fixed multiplicative scheme and 2) an adaptive state-space scheme. We document how inaccuracies in observation uncertainty affect errors in EnKF posteriors and study the combined impacts of misspecified initial observation uncertainty, sampling error, and model error on Desroziers estimates. We find that whether observation uncertainty is over- or underestimated greatly affects the stability of data assimilation and the accuracy of Desroziers estimates and that preference should be given to initial overestimates. Inline estimates of Desroziers tend to remove the dependence between ensemble spread–skill and the initially prescribed observation error. In addition, we find that the inclusion of model error introduces spurious correlations in observation uncertainty estimates. Further, we note that the adaptive inflation scheme is less robust than fixed inflation at mitigating multiple sources of error. Last, sampling error strongly exacerbates existing sources of error and greatly degrades EnKF estimates, which translates into biased Desroziers estimates of observation error covariance.

Significance Statement

To generate accurate predictions of various components of the Earth system, numerical models require an accurate specification of state variables at our current time. This step adopts a probabilistic consideration of our current state estimate versus information provided from environmental measurements of the true state. Various strategies exist for estimating uncertainty in observations within this framework, but are sensitive to a host of assumptions, which are investigated in this study.

more » « less
Author(s) / Creator(s):
 ;  ;  
Publisher / Repository:
American Meteorological Society
Date Published:
Journal Name:
Monthly Weather Review
Page Range / eLocation ID:
p. 2295-2306
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Localization is essential to effectively assimilate satellite radiances in ensemble Kalman filters. However, the vertical location and separation from a model grid point variable for a radiance observation are not well defined, which results in complexities when localizing the impact of radiance observations. An adaptive method is proposed to estimate an effective vertical localization independently for each assimilated channel of every satellite platform. It uses sample correlations between ensemble priors of observations and state variables from a cycling data assimilation to estimate the localization function that minimizes the sampling error. The estimated localization functions are approximated by three localization parameters: the localization width, maximum value, and vertical location of the radiance observations. Adaptively estimated localization parameters are used in assimilation experiments with the National Centers for Environmental Prediction (NCEP) Global Forecast System (GFS) model and the National Oceanic and Atmospheric Administration (NOAA) operational ensemble Kalman filter (EnKF). Results show that using the adaptive localization width and vertical location for radiance observations is more beneficial than also including the maximum localization value. The experiment using the adaptively estimated localization width and vertical location performs better than the default Gaspari and Cohn (GC) experiment, and produces similar errors to the optimal GC experiment. The adaptive localization parameters can be computed during the assimilation procedure, so the computational cost needed to tune the optimal GC localization width is saved.

    more » « less
  2. Abstract

    Lack of high‐resolution observations in the inner‐core of tropical cyclones remains a key issue when constructing an accurate initial state of the storm structure. The major implication of an improper initial state is the poor predictability of the future state of the storm. The size and associated hazard from strong winds at the inner‐core make it impossible to sample this region entirely. However, targeting regions of the inner‐core where forecasted atmospheric measurements have high uncertainty can significantly improve the accuracy of measurements for the initial state of the storm. This study provides a scheme for targeted high‐resolution observations for small Unmanned Aircraft Systems (sUAS) platforms (e.g., Coyote sUAS) to improve the estimates of the atmospheric measurement in the inner‐core structure. The benefit of observation is calculated based on the high‐fidelity state‐of‐the‐art hurricane ensemble data assimilation system. Potential locations with the mostinformativemeasurements are identified through exploration of various simulation‐based solutions depending on the state variables (e.g., pressure, temperature, wind speed, relative humidity) and a combined representation of those variables. A sampling‐based sUAS path planning algorithm considers energy usage when locating the regions of highly uncertain prediction of measurements, allowing sUAS to maximize the benefit of observation. Robustness analysis of our algorithm for multiple scenarios of sUAS drop and goal locations shows satisfactory performance against benchmark similar to current NOAA field campaign. With optimized sUAS observations, a data assimilation analysis shows significant improvements of up to 4% in the tropical cyclone structure estimates after resolving uncertainties at targeted locations.

    more » « less
  3. Abstract

    This article presents a novel approach to couple a deterministic four‐dimensional variational (4DVAR) assimilation method with the particle filter (PF) ensemble data assimilation system, to produce a robust approach for dual‐state‐parameter estimation. In our proposed method, the Hybrid Ensemble and Variational Data Assimilation framework for Environmental systems (HEAVEN), we characterize the model structural uncertainty in addition to model parameter and input uncertainties. The sequential PF is formulated within the 4DVAR system to design a computationally efficient feedback mechanism throughout the assimilation period. In this framework, the 4DVAR optimization produces the maximum a posteriori estimate of state variables at the beginning of the assimilation window without the need to develop the adjoint of the forecast model. The 4DVAR solution is then perturbed by a newly defined prior error covariance matrix to generate an initial condition ensemble for the PF system to provide more accurate and reliable posterior distributions within the same assimilation window. The prior error covariance matrix is updated from one cycle to another over the main assimilation period to account for model structural uncertainty resulting in an improved estimation of posterior distribution. The premise of the presented approach is that it (1) accounts for all sources of uncertainties involved in hydrologic predictions, (2) uses a small ensemble size, and (3) precludes the particle degeneracy and sample impoverishment. The proposed method is applied on a nonlinear hydrologic model and the effectiveness, robustness, and reliability of the method is demonstrated for several river basins across the United States.

    more » « less
  4. Abstract

    Estimation of uncertainties (random error statistics) of radio occultation (RO) observations is important for their effective assimilation in numerical weather prediction (NWP) models. Average uncertainties can be estimated for large samples of RO observations and these statistics may be used for specifying the observation errors in NWP data assimilation. However, the uncertainties of individual RO observations vary, and so using average uncertainty estimates will overestimate the uncertainties of some observations and underestimate those of others, reducing their overall effectiveness in the assimilation. Several parameters associated with RO observations or their atmospheric environments have been proposed to estimate individual RO errors. These include the standard deviation of bending angle (BA) departures from either climatology in the upper stratosphere and lower mesosphere (STDV) or the sample mean between 40 and 60 km (STD4060), the local spectral width (LSW), and the magnitude of the horizontal gradient of refractivity (|∇HN|). In this paper we show how the uncertainties of two RO datasets, COSMIC-2 and Spire BA, as well as their combination, vary with these parameters. We find that the uncertainties are highly correlated with STDV and STD4060 in the stratosphere, and with LSW and |∇HN| in the lower troposphere. These results suggest a hybrid error model for individual BA observations that uses an average statistical model of RO errors modified by STDV or STD4060 above 30 km, and LSW or |∇HN| below 8 km.

    Significance Statement

    These results contribute to the understanding of the sources of uncertainties in radio occultation observations. They could be used to improve the effectiveness of these observations in their assimilation into numerical weather prediction and reanalysis models by improving the estimation of their observational errors.

    more » « less
  5. null (Ed.)
    Modelling of cardiac electrical behaviour has led to important mechanistic insights, but important challenges, including uncertainty in model formulations and parameter values, make it difficult to obtain quantitatively accurate results. An alternative approach is combining models with observations from experiments to produce a data-informed reconstruction of system states over time. Here, we extend our earlier data-assimilation studies using an ensemble Kalman filter to reconstruct a three-dimensional time series of states with complex spatio-temporal dynamics using only surface observations of voltage. We consider the effects of several algorithmic and model parameters on the accuracy of reconstructions of known scroll-wave truth states using synthetic observations. In particular, we study the algorithm’s sensitivity to parameters governing different parts of the process and its robustness to several model-error conditions. We find that the algorithm can achieve an acceptable level of error in many cases, with the weakest performance occurring for model-error cases and more extreme parameter regimes with more complex dynamics. Analysis of the poorest-performing cases indicates an initial decrease in error followed by an increase when the ensemble spread is reduced. Our results suggest avenues for further improvement through increasing ensemble spread by incorporating additive inflation or using a parameter or multi-model ensemble. This article is part of the theme issue ‘Uncertainty quantification in cardiac and cardiovascular modelling and simulation’. 
    more » « less