An official website of the United States government
Abstract Pure artificial intelligence (AI)-based weather prediction (AIWP) models have made waves within the scientific community and the media, claiming superior performance to numerical weather prediction (NWP) models. However, these models often lack impactful output variables such as precipitation. One exception is Google DeepMind’s GraphCast model, which became the first mainstream AIWP model to predict precipitation, but performed only limited verification. We present an analysis of the ECMWF’s Integrated Forecasting System (IFS)-initialized (GRAPIFS) and the NCEP’s Global Forecast System (GFS)-initialized (GRAPGFS) GraphCast precipitation forecasts over the contiguous United States and compare to results from the GFS and IFS models using 1) grid-based, 2) neighborhood, and 3) object-oriented metrics verified against the fifth major global reanalysis produced by ECMWF (ERA5) and the NCEP/Environmental Modeling Center (EMC) stage IV precipitation analysis datasets. We affirmed that GRAPGFSand GRAPIFSperform better than the GFS and IFS in terms of root-mean-square error and stable equitable errors in probability space, but the GFS and IFS precipitation distributions more closely align with the ERA5 and stage IV distributions. Equitable threat score also generally favored GraphCast, particularly for lower accumulation thresholds. Fractions skill score for increasing neighborhood sizes shows greater gains for the GFS and IFS than GraphCast, suggesting the NWP models may have a better handle on intensity but struggle with the location. Object-based verification for GraphCast found positive area biases at low accumulation thresholds and large negative biases at high accumulation thresholds. GRAPGFSsaw similar performance gains to GRAPIFSwhen compared to their NWP counterparts, but initializing with the less familiar GFS conditions appeared to lead to an increase in light precipitation. Significance StatementPure artificial intelligence (AI)-based weather prediction (AIWP) has exploded in popularity with promises of better performance and faster run times than numerical weather prediction (NWP) models. However, less attention has been paid to their capability to predict impactful, sensible weather like precipitation, precipitation type, or specific meteorological features. We seek to address this gap by comparing precipitation forecast performance by an AI model called GraphCast to the Global Forecast System (GFS) and the Integrated Forecasting System (IFS) NWP models. While GraphCast does perform better on many verification metrics, it has some limitations for intense precipitation forecasts. In particular, it less frequently predicts intense precipitation events than the GFS or IFS. Overall, this article emphasizes the promise of AIWP while at the same time stresses the need for robust verification by domain experts.
more »
« less
This content will become publicly available on April 16, 2026
Performance of the Pangu‐Weather Deep Learning Model in Forecasting Tornadic Environments
Abstract The development of deep learning (DL) weather forecasting models has made rapid progress and achieved comparable or better skill than traditional Numerical Weather prediction (NWP) models, which are generally computationally intensive. However, applications of these DL models have yet to be fully explored, including for severe convective events. We evaluate the DL model Pangu‐Weather in forecasting tornadic environments with one‐day lead times using convective available potential energy (CAPE), 0–6 bulk wind difference (BWD6), and 0–3 km storm‐relative helicity (SRH3). We also compare its performance to the National Centers for Environmental Prediction (NCEP)'s Global Forecast System (GFS), a traditional NWP model. Pangu‐Weather generally outperforms GFS in predicting BWD6 and SRH3 at the closest grid point and hour of the storm report. However, Pangu‐Weather tends to underpredict the maximum values of all convective parameters in the 1–2 hr before the storm across the surrounding grid points compared to the GFS.
more »
« less
- Award ID(s):
- 2202526
- PAR ID:
- 10625169
- Publisher / Repository:
- American Geophysical Union
- Date Published:
- Journal Name:
- Geophysical Research Letters
- Volume:
- 52
- Issue:
- 7
- ISSN:
- 0094-8276
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Sierras de Córdoba (Argentina) is characterized by the occurrence of extreme precipitation events during the austral warm season. Heavy precipitation in the region has a large societal impact, causing flash floods. This motivates the forecast performance evaluation of 24-h accumulated precipitation and vertical profiles of atmospheric variables from different numerical weather prediction (NWP) models with the final aim of helping water management in the region. The NWP models evaluated include the Global Forecast System (GFS), which parameterizes convection, and convection-permitting simulations of the Weather Research and Forecasting (WRF) Model configured by three institutions: University of Illinois at Urbana–Champaign (UIUC), Colorado State University (CSU), and National Meteorological Service of Argentina (SMN). These models were verified with daily accumulated precipitation data from rain gauges and soundings during the RELAMPAGO-CACTI field campaign. Generally all configurations of the higher-resolution WRFs outperformed the lower-resolution GFS based on multiple metrics. Among the convection-permitting WRF Models, results varied with respect to rainfall threshold and forecast lead time, but the WRFUIUC mostly performed the best. However, elevation-dependent biases existed among the models that may impact the use of the data for different applications. There is a dry (moist) bias in lower (upper) pressure levels which is most pronounced in the GFS. For Córdoba an overestimation of the northern flow forecasted by the NWP configurations at lower levels was encountered. These results show the importance of convection-permitting forecasts in this region, which should be complementary to the coarser-resolution global model forecasts to help various users and decision-makers.more » « less
-
CoDiCast: Conditional Diffusion Model for Global Weather Forecasting with Uncertainty QuantificationAccurate weather forecasting is critical for science and society. However, existing methods have not achieved the combination of high accuracy, low uncertainty, and high computational efficiency simultaneously. On one hand, traditional numerical weather prediction (NWP) models are computationally intensive because of their complexity. On the other hand, most machine learning-based weather prediction (MLWP) approaches offer efficiency and accuracy but remain deterministic, lacking the ability to capture forecast uncertainty. To tackle these challenges, we propose a conditional diffusion model, CoDiCast, to generate global weather prediction, integrating accuracy and uncertainty quantification at a modest computational cost. The key idea behind the prediction task is to generate realistic weather scenarios at a future time point, conditioned on observations from the recent past. Due to the probabilistic nature of diffusion models, they can be properly applied to capture the uncertainty of weather predictions. Therefore, we accomplish uncertainty quantifications by repeatedly sampling from stochastic Gaussian noise for each initial weather state and running the denoising process multiple times. Experimental results demonstrate that CoDiCast outperforms several existing MLWP methods in accuracy, and is faster than NWP models in inference speed. Our model can generate 6-day global weather forecasts, at 6-hour steps and 5.625-degree latitude-longitude resolutions, for over 5 variables, in about 12 minutes on a commodity A100 GPU machine with 80GB memory. The source code is available at https://github.com/JimengShi/CoDiCast.more » « less
-
Abstract Operational forecast models are necessary for the prediction of weather events in real time. Verification of these models must be performed to assess model skills and areas in need of improvement, particularly with different types of weather events that may occur. Despite the devastating impacts that can be caused by tropical cyclones (TCs) that undergo extratropical transition (ET) and become post-tropical cyclones (PTCs), these storms have not been extensively studied in the context of short-term weather prediction. This study completes the first analysis of the Global Forecast System (GFS) and a preoperational version of the newly operational Hurricane Analysis and Forecast System (HAFS) models in forecasting the occurrence of ET and the rainfall associated with ET storms in the North Atlantic basin. GFS’s skill exceeds that of HAFS in forecasting the occurrence of ET, but HAFS tends to have lower track and rain-rate errors in the fully tropical phase of ET storms’ life cycles. Both models simulate rain rates that are often too high near the storm center and fail to capture the larger area of moderate rain rates that greatly contributes to total rainfall accumulation. The discrepancies in rain rates between the models and Integrated Multi-satellitE Retrievals for GPM (IMERG) could be attributed to the models’ tendency to keep storms too intense and too compact with an overly strong warm core, even throughout the ET process.more » « less
-
To make a future run by renewable energy possible, we must design our power system to seamlessly collect, store, and transport the Earth's naturally occurring flows of energy – namely the sun and the wind. Such a future will require that accurate representations of wind and solar resources and their associated variability permeate power systems planning and operational tools. Practically speaking, we must merge weather and power systems modeling. Although many meteorological phenomena that affect wind and solar power production are well-studied in isolation, no coordinated effort has sought to improve medium- and long-term power systems planning using numerical weather prediction (NWP) models. One modern open-source NWP tool – the weather research and forecasting (WRF) model – offers the complexity and flexibility required to integrate weather prediction with a power systems model in any region. However, there are over one million distinct ways to set up WRF. Here, we present a methodology for optimizing the WRF model physics for forecasting wind power density and solar irradiance using a genetic algorithm. The top five setups created by our algorithm outperform all of the recommended setups. Using the simulation results, we train a random forest model to identify which WRF parameters contribute to the lowest forecast errors and produce plots depicting the performance of key physics options to guide energy researchers in quickly setting up an accurate WRF model.more » « less
