An official website of the United States government
Estimating and disentangling epistemic uncertainty, uncertainty that is reducible with more training data, and aleatoric uncertainty, uncertainty that is inherent to the task at hand, is critically important when applying machine learning to highstakes applications such as medical imaging and weather forecasting. Conditional diffusion models’ breakthrough ability to accurately and efficiently sample from the posterior distribution of a dataset now makes uncertainty estimation conceptually straightforward: One need only train and sample from a large ensemble of diffusion models. Unfortunately, training such an ensemble becomes computationally intractable as the complexity of the model architecture grows. In this work we introduce a new approach to ensembling, hyper-diffusion models (HyperDM), which allows one to accurately estimate both epistemic and aleatoric uncertainty with a single model. Unlike existing single-model uncertainty methods like Monte-Carlo dropout and Bayesian neural networks, HyperDM offers prediction accuracy on par with, and in some cases superior to, multi-model ensembles. Furthermore, our proposed approach scales to modern network architectures such as Attention U-Net and yields more accurate uncertainty estimates compared to existing methods. We validate our method on two distinct real-world tasks: x-ray computed tomography reconstruction and weather temperature forecasting. Source code is publicly available at https://github.com/matthewachan/hyperdm.
more »
« less
This content will become publicly available on September 1, 2026
CoDiCast: Conditional Diffusion Model for Global Weather Forecasting with Uncertainty Quantification
Accurate weather forecasting is critical for science and society. However, existing methods have not achieved the combination of high accuracy, low uncertainty, and high computational efficiency simultaneously. On one hand, traditional numerical weather prediction (NWP) models are computationally intensive because of their complexity. On the other hand, most machine learning-based weather prediction (MLWP) approaches offer efficiency and accuracy but remain deterministic, lacking the ability to capture forecast uncertainty. To tackle these challenges, we propose a conditional diffusion model, CoDiCast, to generate global weather prediction, integrating accuracy and uncertainty quantification at a modest computational cost. The key idea behind the prediction task is to generate realistic weather scenarios at a future time point, conditioned on observations from the recent past. Due to the probabilistic nature of diffusion models, they can be properly applied to capture the uncertainty of weather predictions. Therefore, we accomplish uncertainty quantifications by repeatedly sampling from stochastic Gaussian noise for each initial weather state and running the denoising process multiple times. Experimental results demonstrate that CoDiCast outperforms several existing MLWP methods in accuracy, and is faster than NWP models in inference speed. Our model can generate 6-day global weather forecasts, at 6-hour steps and 5.625-degree latitude-longitude resolutions, for over 5 variables, in about 12 minutes on a commodity A100 GPU machine with 80GB memory. The source code is available at https://github.com/JimengShi/CoDiCast.
more »
« less
- Award ID(s):
- 2118329
- PAR ID:
- 10637206
- Publisher / Repository:
- International Joint Conferences on Artificial Intelligence Organization
- Date Published:
- ISBN:
- 978-1-956792-06-5
- Page Range / eLocation ID:
- 9853 to 9861
- Format(s):
- Medium: X
- Location:
- Montreal, Canada
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract The development of deep learning (DL) weather forecasting models has made rapid progress and achieved comparable or better skill than traditional Numerical Weather prediction (NWP) models, which are generally computationally intensive. However, applications of these DL models have yet to be fully explored, including for severe convective events. We evaluate the DL model Pangu‐Weather in forecasting tornadic environments with one‐day lead times using convective available potential energy (CAPE), 0–6 bulk wind difference (BWD6), and 0–3 km storm‐relative helicity (SRH3). We also compare its performance to the National Centers for Environmental Prediction (NCEP)'s Global Forecast System (GFS), a traditional NWP model. Pangu‐Weather generally outperforms GFS in predicting BWD6 and SRH3 at the closest grid point and hour of the storm report. However, Pangu‐Weather tends to underpredict the maximum values of all convective parameters in the 1–2 hr before the storm across the surrounding grid points compared to the GFS.more » « less
-
Abstract Vertical profiles of temperature and dewpoint are useful in predicting deep convection that leads to severe weather, which threatens property and lives. Currently, forecasters rely on observations from radiosonde launches and numerical weather prediction (NWP) models. Radiosonde observations are, however, temporally and spatially sparse, and NWP models contain inherent errors that influence short-term predictions of high impact events. This work explores using machine learning (ML) to postprocess NWP model forecasts, combining them with satellite data to improve vertical profiles of temperature and dewpoint. We focus on different ML architectures, loss functions, and input features to optimize predictions. Because we are predicting vertical profiles at 256 levels in the atmosphere, this work provides a unique perspective at using ML for 1D tasks. Compared to baseline profiles from the Rapid Refresh (RAP), ML predictions offer the largest improvement for dewpoint, particularly in the middle and upper atmosphere. Temperature improvements are modest, but CAPE values are improved by up to 40%. Feature importance analyses indicate that the ML models are primarily improving incoming RAP biases. While additional model and satellite data offer some improvement to the predictions, architecture choice is more important than feature selection in fine-tuning the results. Our proposed deep residual U-Net performs the best by leveraging spatial context from the input RAP profiles; however, the results are remarkably robust across model architecture. Further, uncertainty estimates for every level are well calibrated and can provide useful information to forecasters.more » « less
-
Abstract Pure artificial intelligence (AI)-based weather prediction (AIWP) models have made waves within the scientific community and the media, claiming superior performance to numerical weather prediction (NWP) models. However, these models often lack impactful output variables such as precipitation. One exception is Google DeepMind’s GraphCast model, which became the first mainstream AIWP model to predict precipitation, but performed only limited verification. We present an analysis of the ECMWF’s Integrated Forecasting System (IFS)-initialized (GRAPIFS) and the NCEP’s Global Forecast System (GFS)-initialized (GRAPGFS) GraphCast precipitation forecasts over the contiguous United States and compare to results from the GFS and IFS models using 1) grid-based, 2) neighborhood, and 3) object-oriented metrics verified against the fifth major global reanalysis produced by ECMWF (ERA5) and the NCEP/Environmental Modeling Center (EMC) stage IV precipitation analysis datasets. We affirmed that GRAPGFSand GRAPIFSperform better than the GFS and IFS in terms of root-mean-square error and stable equitable errors in probability space, but the GFS and IFS precipitation distributions more closely align with the ERA5 and stage IV distributions. Equitable threat score also generally favored GraphCast, particularly for lower accumulation thresholds. Fractions skill score for increasing neighborhood sizes shows greater gains for the GFS and IFS than GraphCast, suggesting the NWP models may have a better handle on intensity but struggle with the location. Object-based verification for GraphCast found positive area biases at low accumulation thresholds and large negative biases at high accumulation thresholds. GRAPGFSsaw similar performance gains to GRAPIFSwhen compared to their NWP counterparts, but initializing with the less familiar GFS conditions appeared to lead to an increase in light precipitation. Significance StatementPure artificial intelligence (AI)-based weather prediction (AIWP) has exploded in popularity with promises of better performance and faster run times than numerical weather prediction (NWP) models. However, less attention has been paid to their capability to predict impactful, sensible weather like precipitation, precipitation type, or specific meteorological features. We seek to address this gap by comparing precipitation forecast performance by an AI model called GraphCast to the Global Forecast System (GFS) and the Integrated Forecasting System (IFS) NWP models. While GraphCast does perform better on many verification metrics, it has some limitations for intense precipitation forecasts. In particular, it less frequently predicts intense precipitation events than the GFS or IFS. Overall, this article emphasizes the promise of AIWP while at the same time stresses the need for robust verification by domain experts.more » « less
-
Abstract To address Objective II of the National Space Weather Strategy and Action Plan “Develop and Disseminate Accurate and Timely Space Weather Characterization and Forecasts” and US Congress PROSWIFT Act 116–181, our team is developing a new set of open-source software that would ensure substantial improvements of Space Weather (SWx) predictions. On the one hand, our focus is on the development of data-driven solar wind models. On the other hand, each individual component of our software is designed to have accuracy higher than any existing SWx prediction tools with a dramatically improved performance. This is done by the application of new computational technologies and enhanced data sources. The development of such software paves way for improved SWx predictions accompanied with an appropriate uncertainty quantification. This makes it possible to forecast hazardous SWx effects on the space-borne and ground-based technological systems, and on human health. Our models include (1) a new, open-source solar magnetic flux model (OFT), which evolves information to the back side of the Sun and its poles, and updates the model flux with new observations using data assimilation methods; (2) a new potential field solver (POT3D) associated with the Wang–Sheeley–Arge coronal model, and (3) a new adaptive, 4-th order of accuracy solver (HelioCubed) for the Reynolds-averaged MHD equations implemented on mapped multiblock grids (cubed spheres). We describe the software and results obtained with it, including the application of machine learning to modeling coronal mass ejections, which makes it possible to improve SWx predictions by decreasing the time-of-arrival mismatch. The tests show that our software is formally more accurate and performs much faster than its predecessors used for SWx predictions.more » « less
