An official website of the United States government
Introduction:There is increasing interest in developing mathematical and computational models to forecast adverse events in physiological systems. Examples include falls, the onset of fatal cardiac arrhythmias, and adverse surgical outcomes. However, the dynamics of physiological systems are known to be exceedingly complex and perhaps even chaotic. Since no model can be perfect, it becomes important to understand how forecasting can be improved, especially when training data is limited. An adverse event that can be readily studied in the laboratory is the occurrence of stick falls when humans attempt to balance a stick on their fingertips. Over the last 20 years, this task has been extensively investigated experimentally, and presently detailed mathematical models are available. Methods:Here we use a long short-term memory (LTSM) deep learning network to forecast stick falls. We train this model to forecast stick falls in three ways: 1) using only data generated by the mathematical model (synthetic data), 2) using only stick balancing recordings of stick falls measured using high-speed motion capture measurements (human data), and 3) using transfer learning which combines a model trained using synthetic data plus a small amount of human balancing data. Results:We observe that the LTSM model is much more successful in forecasting a fall using synthetic data than it is in forecasting falls for models trained with limited available human data. However, with transfer learning, i.e., the LTSM model pre-trained with synthetic data and re-trained with a small amount of real human balancing data, the ability to forecast impending falls in human data is vastly improved. Indeed, it becomes possible to correctly forecast 60%–70% of real human stick falls up to 2.35 s in advance. Conclusion:These observations support the use of model-generated data and transfer learning techniques to improve the ability of computational models to forecast adverse physiological events.
more »
« less
Long-term stability and generalization of observationally-constrained stochastic data-driven models for geophysical turbulence
Abstract Recent years have seen a surge in interest in building deep learning-based fully data-driven models for weather prediction. Such deep learning models, if trained on observations can mitigate certain biases in current state-of-the-art weather models, some of which stem from inaccurate representation of subgrid-scale processes. However, these data-driven models, being over-parameterized, require a lot of training data which may not be available from reanalysis (observational data) products. Moreover, an accurate, noise-free, initial condition to start forecasting with a data-driven weather model is not available in realistic scenarios. Finally, deterministic data-driven forecasting models suffer from issues with long-term stability and unphysical climate drift, which makes these data-driven models unsuitable for computing climate statistics. Given these challenges, previous studies have tried to pre-train deep learning-based weather forecasting models on a large amount of imperfect long-term climate model simulations and then re-train them on available observational data. In this article, we propose a convolutional variational autoencoder (VAE)-based stochastic data-driven model that is pre-trained on an imperfect climate model simulation from a two-layer quasi-geostrophic flow and re-trained, using transfer learning, on a small number of noisy observations from a perfect simulation. This re-trained model then performs stochastic forecasting with a noisy initial condition sampled from the perfect simulation. We show that our ensemble-based stochastic data-driven model outperforms a baseline deterministic encoder–decoder-based convolutional model in terms of short-term skills, while remaining stable for long-term climate simulations yielding accurate climatology.
more »
« less
- Award ID(s):
- 2005123
- PAR ID:
- 10472804
- Publisher / Repository:
- Cambridge University
- Date Published:
- Journal Name:
- Environmental Data Science
- Volume:
- 2
- ISSN:
- 2634-4602
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Accurate and timely regional weather prediction is vital for sectors dependent on weather-related decisions. Traditional prediction methods, based on atmospheric equations, often struggle with coarse temporal resolutions and inaccuracies. This article presents a novel machine learning (ML) model, called Micro–Macro (MiMa), that integrates both near-surface obser- vational data from Kentucky Mesonet stations (collected every 5 min, known as Micro data) and hourly atmospheric numerical outputs (termed as Macro data) for fine-resolution weather forecasting. The MiMa model employs an encoder–decoder trans- former structure, with two encoders for processing multivariate data from both datasets and a decoder for forecasting weather variables over short time horizons. Each instance of the MiMa model, called a modelet, predicts the values of a specific weather parameter at an individual mesonet station. The approach is extended with Regional MiMa (Re-MiMa) modelets, which are designed to predict weather variables at ungauged locations by training on multivariate data from a few representative stations in a region, tagged with their elevations. Re-MiMa can provide highly accurate predictions across an entire region, even in areas without observational stations. Experimental results show that MiMa significantly outperforms current models, with Re-MiMa offering precise short-term forecasts for ungauged locations, marking a significant advancement in weather forecasting accu- racy and applicability.more » « less
-
Abstract Two key challenges in the development of data‐driven gravity‐wave parameterizations are generalization, how to ensure that a data‐driven scheme trained on the present‐day climate will continue to work in a new climate regime, and calibration, how to account for biases in the “host” climate model. Both problems depend fundamentally on the response to out‐of‐sample inputs compared with the training dataset, and are often conflicting. The ability to generalize to new climate regimes often goes hand in hand with sensitivity to model biases. To probe these challenges, we employ a one‐dimensional (1D) quasibiennial oscillation (QBO) model with a stochastic source term that represents convectively generated gravity waves in the Tropics with randomly varying strengths and spectra. We employ an array of machine‐learning models consisting of a fully connected feed‐forward neural network, a dilated convolutional neural network, an encoder–decoder, a boosted forest, and a support‐vector regression model. Our results demonstrate that data‐driven schemes trained on “observations” can be critically sensitive to model biases in the wave sources. While able to emulate accurately the stochastic source term on which they were trained, all of our schemes fail to simulate fully the expected QBO period or amplitude, even with the slightest perturbation to the wave sources. The main takeaway is that some measures will always be required to ensure the proper response to climate change and to account for model biases. We examine one approach based on the ideas of optimal transport, where the wave sources in the model are first remapped to the observed one before applying the data‐driven scheme. This approach is agnostic to the data‐driven method and guarantees that the model adheres to the observational constraints, making sure the model yields the right results for the right reasons.more » « less
-
Abstract We present a lightweight, easy‐to‐train, low‐resolution, fully data‐driven climate emulator, LUCIE, that can be trained on as low as 2 years of 6‐hourly ERA5 data. Unlike most state‐of‐the‐art AI weather models, LUCIE remains stable and physically consistent for 100 years of autoregressive simulation with 100 ensemble members. Long‐term mean climatology from LUCIE's simulation of temperature, wind, precipitation, and humidity matches that of ERA5 data, along with the variability. We further demonstrate how well extreme weather events and their return periods can be estimated from a large ensemble of long‐term simulations. We further discuss an improved training strategy with a hard‐constrained first‐order integrator to suppress autoregressive error growth, a novel spectral regularization strategy to better capture fine‐scale dynamics, and finally an optimization algorithm that enables data‐limited (as low as 2 years of 6‐hourly data) training of the emulator without losing stability and physical consistency. Finally, we provide a scaling experiment to compare the long‐term bias of LUCIE with respect to the number of training samples. Importantly, LUCIE is an easy to use model that can be trained in just 2.4 hr on a single A‐100 GPU, allowing for multiple experiments that can explore important scientific questions that could be answered with large ensembles of long‐term simulations, for example, the impact of different variables on the simulation, dynamic response to external forcing, and estimation of extreme weather events, amongst others.more » « less
-
According to twenty-first century climate-model projections, greenhouse warming will intensify rainfall variability and extremes across the globe. However, verifying this prediction using observations has remained a substantial challenge owing to large natural rainfall fluctuations at regional scales. Here we show that deep learning successfully detects the emerging climate-change signals in daily precipitation fields during the observed record. We trained a convolutional neural network (CNN) with daily precipitation fields and annual global mean surface air temperature data obtained from an ensemble of present-day and future climate-model simulations. After applying the algorithm to the observational record, we found that the daily precipitation data represented an excellent predictor for the observed planetary warming, as they showed a clear deviation from natural variability since the mid-2010s. Furthermore, we analysed the deep-learning model with an explainable framework and observed that the precipitation variability of the weather timescale (period less than 10 days) over the tropical eastern Pacific and mid-latitude storm-track regions was most sensitive to anthropogenic warming. Our results highlight that, although the long-term shifts in annual mean precipitation remain indiscernible from the natural background variability, the impact of global warming on daily hydrological fluctuations has already emerged.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1017/eds.2022.30
