An official website of the United States government
Nowcasts, or near-real-time (NRT) forecasts, of soil moisture based on the Soil Moisture Active and Passive (SMAP) mission could provide substantial value for a range of applications including hazards monitoring and agricultural planning. To provide such a NRT forecast with high fidelity, we enhanced a time series deep learning architecture, long short-term memory (LSTM), with a novel data integration (DI) kernel to assimilate the most recent SMAP observations as soon as they become available. The kernel is adaptive in that it can accommodate irregular observational schedules. Testing over the CONUS, this NRT forecast product showcases predictions with unprecedented accuracy when evaluated against subsequent SMAP retrievals. It showed smaller error than NRT forecasts reported in the literature, especially at longer forecast latency. The comparative advantage was due to LSTM’s structural improvements, as well as its ability to utilize more input variables and more training data. The DI-LSTM was compared to the original LSTM model that runs without data integration, referred to as the projection model here. We found that the DI procedure removed the autocorrelated effects of forcing errors and errors due to processes not represented in the inputs, for example, irrigation and floodplain/lake inundation, as well as mismatches due to unseen forcing conditions. The effects of this purely data-driven DI kernel are discussed for the first time in the geosciences. Furthermore, this work presents an upper-bound estimate for the random component of the SMAP retrieval error.
more »
« less
Evaluating the Potential and Challenges of an Uncertainty Quantification Method for Long Short‐Term Memory Models for Soil Moisture Predictions
Abstract Recently, recurrent deep networks have shown promise to harness newly available satellite‐sensed data for long‐term soil moisture projections. However, to be useful in forecasting, deep networks must also provide uncertainty estimates. Here we evaluated Monte Carlo dropout with an input‐dependent data noise term (MCD+N), an efficient uncertainty estimation framework originally developed in computer vision, for hydrologic time series predictions. MCD+N simultaneously estimates a heteroscedastic input‐dependent data noise term (a trained error model attributable to observational noise) and a network weight uncertainty term (attributable to insufficiently constrained model parameters). Although MCD+N has appealing features, many heuristic approximations were employed during its derivation, and rigorous evaluations and evidence of its asserted capability to detect dissimilarity were lacking. To address this, we provided an in‐depth evaluation of the scheme's potential and limitations. We showed that for reproducing soil moisture dynamics recorded by the Soil Moisture Active Passive (SMAP) mission, MCD+N indeed gave a good estimate of predictive error, provided that we tuned a hyperparameter and used a representative training data set. The input‐dependent term responded strongly to observational noise, while the model term clearly acted as a detector for physiographic dissimilarity from the training data, behaving as intended. However, when the training and test data were characteristically different, the input‐dependent term could be misled, undermining its reliability. Additionally, due to the data‐driven nature of the model, data noise also influences network weight uncertainty, and therefore the two uncertainty terms are correlated. Overall, this approach has promise, but care is needed to interpret the results.
more »
« less
- Award ID(s):
- 1832294
- PAR ID:
- 10452480
- Publisher / Repository:
- DOI PREFIX: 10.1029
- Date Published:
- Journal Name:
- Water Resources Research
- Volume:
- 56
- Issue:
- 12
- ISSN:
- 0043-1397
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Satellite precipitation products, as all quantitative estimates, come with some inherent degree of uncertainty. To associate a quantitative value of the uncertainty to each individual estimate, error modeling is necessary. Most of the error models proposed so far compute the uncertainty as a function of precipitation intensity only, and only at one specific spatiotemporal scale. We propose a spectral error model that accounts for the neighboring space–time dynamics of precipitation into the uncertainty quantification. Systematic distortions of the precipitation signal and random errors are characterized distinctively in every frequency–wavenumber band in the Fourier domain, to accurately characterize error across scales. The systematic distortions are represented as a deterministic space–time linear filtering term. The random errors are represented as a nonstationary additive noise. The spectral error model is applied to the IMERG multisatellite precipitation product, and its parameters are estimated empirically through a system identification approach using the GV-MRMS gauge–radar measurements as reference (“truth”) over the eastern United States. The filtering term is found to be essentially low-pass (attenuating the fine-scale variability). While traditional error models attribute most of the error variance to random errors, it is found here that the systematic filtering term explains 48% of the error variance at the native resolution of IMERG. This fact confirms that, at high resolution, filtering effects in satellite precipitation products cannot be ignored, and that the error cannot be represented as a purely random additive or multiplicative term. An important consequence is that precipitation estimates derived from different sources shall not be expected to automatically have statistically independent errors. Significance StatementSatellite precipitation products are nowadays widely used for climate and environmental research, water management, risk analysis, and decision support at the local, regional, and global scales. For all these applications, knowledge about the accuracy of the products is critical for their usability. However, products are not systematically provided with a quantitative measure of the uncertainty associated with each individual estimate. Various parametric error models have been proposed for uncertainty quantification, mostly assuming that the uncertainty is only a function of the precipitation intensity at the pixel and time of interest. By projecting satellite precipitation fields and their retrieval errors into the Fourier frequency–wavenumber domain, we show that we can explicitly take into account the neighboring space–time multiscale dynamics of precipitation and compute a scale-dependent uncertainty.more » « less
-
Abstract The Consistent Artificial Intelligence (AI)-based Soil Moisture (CASM) dataset is a global, consistent, and long-term, remote sensing soil moisture (SM) dataset created using machine learning. It is based on the NASA Soil Moisture Active Passive (SMAP) satellite mission SM data and is aimed at extrapolating SMAP-like quality SM back in time using previous satellite microwave platforms. CASM represents SM in the top soil layer, and it is defined on a global 25 km EASE-2 grid and for 2002–2020 with a 3-day temporal resolution. The seasonal cycle is removed for the neural network training to ensure its skill is targeted at predicting SM extremes. CASM comparison to 367 globalin-situSM monitoring sites shows a SMAP-like median correlation of 0.66. Additionally, the SM product uncertainty was assessed, and both aleatoric and epistemic uncertainties were estimated and included in the dataset. CASM dataset can be used to study a wide range of hydrological, carbon cycle, and energy processes since only a consistent long-term dataset allows assessing changes in water availability and water stress.more » « less
-
Abstract Soil moisture and evapotranspiration (ET) are important components of boreal forest hydrology that affect ecological processes and land‐atmosphere feedbacks. Future trends in soil moisture in particular are uncertain. Therefore, accurate modeling of these dynamics and understanding of concomitant sources of uncertainty are critical. Here, we conduct a global sensitivity analysis, Monte Carlo parameterization, and analysis of parameter uncertainty and its contribution to future soil moisture and ET uncertainty using a physically based ecohydrologic model in multiple boreal forest types. Soil and plant hydraulic parameters and LAI have the largest effects on simulated summer soil moisture at two contrasting sites. In future scenario simulations, the selection of parameters and global climate model (GCM) choice between two GCMs influence projected changes in soil moisture and ET about as much as the projected effects of climate change in the less sensitive GCM with a late‐century, high‐emissions scenario, though the relative effects of parameters, GCM, and climate vary among hydrologic variables and study sites. Saturated volumetric water content and sensitivity of stomatal conductance to vapor pressure deficit have the most statistically significant effects on change in ET and soil moisture, though there is considerable variability between sites and GCMs. The results of this study provide estimates of: (a) parameter importance and statistical significance for soil moisture modeling, (b) parameter values for physically based soil‐vegetation‐atmosphere transfer models in multiple boreal forest types, and (c) the contributions of uncertainty in these parameters to soil moisture and ET uncertainty in future climates.more » « less
-
Identifying which aspects of global environmental change are driving observed ecosystem process responses is a great challenge. Here, we address how long-term (10-25 year) alterations in soil moisture, and nitrogen (N) oligotrophication (i.e. decreases in soil N availability relative to plant demand), alter the production of plant-available N via net mineralization and nitrification in a northern hardwood forest. Our objectives were to determine whether soil moisture has changed over the past decade and whether N cycle processes have become less sensitive to soil moisture over time due to N oligotrophication. We used long-term data sets from several related studies to show: (i) increasing winter soil temperatures and declining summer soil moisture from late 2010 into 2024; (ii) reductions in sensitivity of N cycling rates to soil moisture, and (iii) declining moisture-adjusted N cycle processes (the ratio of rate of N process:soil moisture) over time in both summer and winter. These changes suggest continued reductions in N availability to plants in these forests, with potential effects on forest productivity and response to disturbance.more » « less
