skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Advancing interpretability of machine-learning prediction models
Abstract This paper proposes an approach to diagnosing the skill of a machine-learning prediction model based on finding combinations of variables that minimize the normalized mean square error of the predictions. This technique is attractive because it compresses the positive skill of a forecast model into the smallest number of components. The resulting components can then be analyzed much like principal components, including the construction of regression maps for investigating sources of skill. The technique is illustrated with a machine-learning model of week 3–4 predictions of western US wintertime surface temperatures. The technique reveals at least two patterns of large-scale temperature variations that are skillfully predicted. The predictability of these patterns is generally consistent between climate model simulations and observations. The predictability is determined largely by sea surface temperature variations in the Pacific, particularly the region associated with the El Nino-Southern Oscillation. This result is not surprising, but the fact that it emerges naturally from the technique demonstrates that the technique can be helpful in “explaining” the source of predictability in machine-learning models.  more » « less
Award ID(s):
1822221
PAR ID:
10431068
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Environmental Data Science
Volume:
1
ISSN:
2634-4602
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; (, Frontiers in Climate)
    null (Ed.)
    This paper shows that skillful week 3–4 predictions of a large-scale pattern of 2 m temperature over the US can be made based on the Nino3.4 index alone, where skillful is defined to be better than climatology. To find more skillful regression models, this paper explores various machine learning strategies (e.g., ridge regression and lasso), including those trained on observations and on climate model output. It is found that regression models trained on climate model output yield more skillful predictions than regression models trained on observations, presumably because of the larger training sample. Nevertheless, the skill of the best machine learning models are only modestly better than ordinary least squares based on the Nino3.4 index. Importantly, this fact is difficult to infer from the parameters of the machine learning model because very different parameter sets can produce virtually identical predictions. For this reason, attempts to interpret the source of predictability from the machine learning model can be very misleading. The skill of machine learning models also are compared to those of a fully coupled dynamical model, CFSv2. The results depend on the skill measure: for mean square error, the dynamical model is slightly worse than the machine learning models; for correlation skill, the dynamical model is only modestly better than machine learning models or the Nino3.4 index. In summary, the best predictions of the large-scale pattern come from machine learning models trained on long climate simulations, but the skill is only modestly better than predictions based on the Nino3.4 index alone. 
    more » « less
  2. ; ; (, Environmental Research Letters)
    Abstract An open question in the study of climate prediction is whether internal variability will continue to contribute to prediction skill in the coming decades, or whether predictable signals will be overwhelmed by rising temperatures driven by anthropogenic forcing. We design a neural network that is interpretable such that its predictions can be decomposed to examine the relative contributions of external forcing and internal variability to future regional sea surface temperature (SST) trend predictions in the near-term climate (2020–2050). We show that there is additional prediction skill to be garnered from internal variability in the Community Earth System Model version 2 Large Ensemble, even in a relatively high forcing future scenario. This predictability is especially apparent in the North Atlantic, North Pacific and Tropical Pacific Oceans as well as in the Southern Ocean. We further investigate how prediction skill covaries across the ocean and find three regions with distinct coherent prediction skill driven by internal variability. SST trend predictability is found to be associated with consistent patterns of decadal variability for the grid points within each region. 
    more » « less
  3. ; ; (, Geophysical Research Letters)
    Abstract We use neural networks and large climate model ensembles to explore predictability of internal variability in sea surface temperature (SST) anomalies on interannual (1–3 years) and decadal (1–5 and 3–7 years) timescales. We find that neural networks can skillfully predict SST anomalies at these lead times, especially in the North Atlantic, North Pacific, Tropical Pacific, Tropical Atlantic and Southern Ocean. The spatial patterns of SST predictability vary across the nine climate models studied. The neural networks identify “windows of opportunity” where future SST anomalies can be predicted with more certainty. Neural networks trained on climate models also make skillful SST predictions in reconstructed observations, although the skill varies depending on which climate model the network was trained. Our results highlight that neural networks can identify predictable internal variability within existing climate data sets and show important differences in how well patterns of SST predictability in climate models translate to the real world. 
    more » « less
  4. ; ; (, International Journal of Climatology)
    Abstract This study illustrates the considerable improvement in accuracy achievable for long‐lead forecasts (18 months) of the Ocean Niño Index (ONI) through the utilization of a long short‐term memory (LSTM) machine learning algorithm. The research assesses the predictive potential of eight predictors from both tropical and extratropical regions constructed based on sea surface temperature, outgoing longwave radiation, sea surface height and zonal and meridional wind anomalies. In comparison to linear regression model forecasts, the LSTM model outperforms them for both the tropical and extratropical predictor sets. Among all the predictors, the western North Pacific (WNP) index demonstrates the highest prediction skill in ONI forecasts, followed by the North Tropical Atlantic (NTA) index and then the sea surface height index. While other predictors help the LSTM model to forecast either the phase variation of the amplitude variation of the observed ONI, the extratropical WNP predictor enables the LSTM model to forecast both variations. This superiority can be attributed to the involvement of SST anomalies in the WNP region in both tropical and extratropical El Niño–Southern Oscillation (ENSO) dynamics, allowing for the utilization of predictive potential from both components of ENSO dynamics. The study also concludes that the extratropical ENSO dynamics provide a robust source of predictability for long‐lead ENSO forecasts, which can be effectively harnessed using the LSTM model. 
    more » « less
  5. ; (, Atmosphere)
    Irrigation can affect climate and weather patterns from regional to global scales through the alteration of surface water and energy balances. Here, we couple a land-surface model (LSM) that includes various human land-water management activities including irrigation with an atmospheric general circulation model (AGCM) to examine the impacts of irrigation-induced land disturbance on the subseasonal predictability of near-surface variables. Results indicate that the simulated global irrigation and groundwater withdrawals (circa 2000) are ~3600 and ~370 km3/year, respectively, which are in good agreement with previous estimates from country statistics and offline–LSMs. Subseasonal predictions for boreal summers during the 1986–1995 period suggest that the spread among ensemble simulations of air temperature can be substantially reduced by using realistic land initializations considering irrigation-induced changes in soil moisture. Additionally, it is found that the subseasonal forecast skill for near-surface temperature and sea level pressure significantly improves when human-induced land disturbance is accounted for in the AGCM. These results underscore the need to incorporate irrigation into weather forecast models, such as the global forecast system. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email