skip to main content


Title: Using Machine Learning to Generate Storm-Scale Probabilistic Guidance of Severe Weather Hazards in the Warn-on-Forecast System
Abstract A primary goal of the National Oceanic and Atmospheric Administration Warn-on-Forecast (WoF) project is to provide rapidly updating probabilistic guidance to human forecasters for short-term (e.g., 0–3 h) severe weather forecasts. Postprocessing is required to maximize the usefulness of probabilistic guidance from an ensemble of convection-allowing model forecasts. Machine learning (ML) models have become popular methods for postprocessing severe weather guidance since they can leverage numerous variables to discover useful patterns in complex datasets. In this study, we develop and evaluate a series of ML models to produce calibrated, probabilistic severe weather guidance from WoF System (WoFS) output. Our dataset includes WoFS ensemble forecasts available every 5 min out to 150 min of lead time from the 2017–19 NOAA Hazardous Weather Testbed Spring Forecasting Experiments (81 dates). Using a novel ensemble storm-track identification method, we extracted three sets of predictors from the WoFS forecasts: intrastorm state variables, near-storm environment variables, and morphological attributes of the ensemble storm tracks. We then trained random forests, gradient-boosted trees, and logistic regression algorithms to predict which WoFS 30-min ensemble storm tracks will overlap a tornado, severe hail, and/or severe wind report. To provide rigorous baselines against which to evaluate the skill of the ML models, we extracted the ensemble probabilities of hazard-relevant WoFS variables exceeding tuned thresholds from each ensemble storm track. The three ML algorithms discriminated well for all three hazards and produced more reliable probabilities than the baseline predictions. Overall, the results suggest that ML-based postprocessing of dynamical ensemble output can improve short-term, storm-scale severe weather probabilistic guidance.  more » « less
Award ID(s):
2019758
NSF-PAR ID:
10422706
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
Monthly Weather Review
Volume:
149
Issue:
5
ISSN:
0027-0644
Page Range / eLocation ID:
1535 to 1557
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    An ensemble postprocessing method is developed for the probabilistic prediction of severe weather (tornadoes, hail, and wind gusts) over the conterminous United States (CONUS). The method combines conditional generative adversarial networks (CGANs), a type of deep generative model, with a convolutional neural network (CNN) to postprocess convection-allowing model (CAM) forecasts. The CGANs are designed to create synthetic ensemble members from deterministic CAM forecasts, and their outputs are processed by the CNN to estimate the probability of severe weather. The method is tested using High-Resolution Rapid Refresh (HRRR) 1–24-h forecasts as inputs and Storm Prediction Center (SPC) severe weather reports as targets. The method produced skillful predictions with up to 20% Brier skill score (BSS) increases compared to other neural-network-based reference methods using a testing dataset of HRRR forecasts in 2021. For the evaluation of uncertainty quantification, the method is overconfident but produces meaningful ensemble spreads that can distinguish good and bad forecasts. The quality of CGAN outputs is also evaluated. Results show that the CGAN outputs behave similarly to a numerical ensemble; they preserved the intervariable correlations and the contribution of influential predictors as in the original HRRR forecasts. This work provides a novel approach to postprocess CAM output using neural networks that can be applied to severe weather prediction.

    Significance Statement

    We use a new machine learning (ML) technique to generate probabilistic forecasts of convective weather hazards, such as tornadoes and hailstorms, with the output from high-resolution numerical weather model forecasts. The new ML system generates an ensemble of synthetic forecast fields from a single forecast, which are then used to train ML models for convective hazard prediction. Using this ML-generated ensemble for training leads to improvements of 10%–20% in severe weather forecast skills compared to using other ML algorithms that use only output from the single forecast. This work is unique in that it explores the use of ML methods for producing synthetic forecasts of convective storm events and using these to train ML systems for high-impact convective weather prediction.

     
    more » « less
  2. Abstract Observational data collection is extremely hazardous in supercell storm environments, which makes for a scarcity of data used for evaluating the storm-scale guidance from convection allowing models (CAMs) like the National Oceanic and Atmospheric Administration (NOAA) Warn-on-Forecast System (WoFS). The Targeted Observations with UAS and Radar of Supercells (TORUS) 2019 field mission provided a rare opportunity to not only collect these observations, but to do so with advanced technology: vertically pointing Doppler lidar. One standing question for WoFS is how the system forecasts the feedback between supercells and their near-storm environment. The lidar can observe vertical profiles of wind over time, creating unique datasets to compare to WoFS kinematic predictions in rapidly evolving severe weather environments. Mobile radiosonde data are also presented to provide a thermodynamic comparison. The five lidar deployments (three of which observed tornadic supercells) analyzed show WoFS accurately predicted general kinematic trends in the inflow environment; however, the predicted feedback between the supercell and its environment, which resulted in enhanced inflow and larger storm-relative helicity (SRH), were muted relative to observations. The radiosonde observations reveal an overprediction of CAPE in WoFS forecasts, both in the near and far field, with an inverse relationship between the CAPE errors and distance from the storm. Significance Statement It is difficult to evaluate the accuracy of weather prediction model forecasts of severe thunderstorms because observations are rarely available near the storms. However, the TORUS 2019 field experiment collected multiple specialized observations in the near-storm environment of supercells, which are compared to the same near-storm environments predicted by the National Oceanic and Atmospheric Administration (NOAA) Warn-on-Forecast System (WoFS) to gauge its performance. Unique to this study is the use of mobile Doppler lidar observations in the evaluation; lidar can retrieve the horizontal winds in the few kilometers above ground on time scales of a few minutes. Using lidar and radiosonde observations in the near-storm environment of three tornadic supercells, we find that WoFS generally predicts the expected trends in the evolution of the near-storm wind profile, but the response is muted compared to observations. We also find an inverse relationship of errors in instability to distance from the storm. These results can aid model developers in refining model physics to better predict severe storms. 
    more » « less
  3. Hail forecast evaluations provide important insight into microphysical treatment of rimed ice. In this study we evaluate explicit 0–90-min EnKF-based storm-scale (500-m horizontal grid spacing) hail forecasts for a severe weather event that occurred in Oklahoma on 19 May 2013. Forecast ensembles are run using three different bulk microphysics (MP) schemes: the Milbrandt–Yau double-moment scheme (MY2), the Milbrandt–Yau triple-moment scheme (MY3), and the NSSL variable density-rimed ice double-moment scheme (NSSL). Output from a hydrometeor classification algorithm is used to verify surface hail size forecasts. All three schemes produce forecasts that predict the coverage of severe surface hail with moderate to high skill, but exhibit less skill at predicting significant severe hail coverage. A microphysical budget analysis is conducted to better understand hail growth processes in all three schemes. The NSSL scheme uses two-variable density-rimed ice categories to create large hailstones from dense, wet growth graupel particles; however, it is noted the scheme underestimates the coverage of significant severe hail. Both the MY2 and MY3 schemes produce many small hailstones aloft from unrimed, frozen raindrops; in the melting layer, hailstones become much larger than observations because of the excessive accretion of water. The results of this work highlight the importance of using a MP scheme that realistically models microphysical processes.

     
    more » « less
  4. Abstract

    The quantification of storm updrafts remains unavailable for operational forecasting despite their inherent importance to convection and its associated severe weather hazards. Updraft proxies, like overshooting top area from satellite images, have been linked to severe weather hazards but only relate to a limited portion of the total storm updraft. This study investigates if a machine learning model, namely, U-Nets, can skillfully retrieve maximum vertical velocity and its areal extent from three-dimensional gridded radar reflectivity alone. The machine learning model is trained using simulated radar reflectivity and vertical velocity from the National Severe Storm Laboratory’s convection permitting Warn-on-Forecast System (WoFS). A parametric regression technique using the sinh–arcsinh–normal distribution is adapted to run with U-Nets, allowing for both deterministic and probabilistic predictions of maximum vertical velocity. The best models after hyperparameter search provided less than 50% root mean squared error, a coefficient of determination greater than 0.65, and an intersection over union (IoU) of more than 0.45 on the independent test set composed of WoFS data. Beyond the WoFS analysis, a case study was conducted using real radar data and corresponding dual-Doppler analyses of vertical velocity within a supercell. The U-Net consistently underestimates the dual-Doppler updraft speed estimates by 50%. Meanwhile, the area of the 5 and 10 m s−1updraft cores shows an IoU of 0.25. While the above statistics are not exceptional, the machine learning model enables quick distillation of 3D radar data that is related to the maximum vertical velocity, which could be useful in assessing a storm’s severe potential.

    Significance Statement

    All convective storm hazards (tornadoes, hail, heavy rain, straight line winds) can be related to a storm’s updraft. Yet, there is no direct measurement of updraft speed or area available for forecasters to make their warning decisions from. This paper addresses the lack of observational data by providing a machine learning solution that skillfully estimates the maximum updraft speed within storms from only the radar reflectivity 3D structure. After further vetting the machine learning solutions on additional real-world examples, the estimated storm updrafts will hopefully provide forecasters with an added tool to help diagnose a storm’s hazard potential more accurately.

     
    more » « less
  5. Abstract

    In this study, a new lightning data assimilation (LDA) scheme using Geostationary Lightning Mapper (GLM) flash extent density (FED) is developed and implemented in the National Severe Storms Laboratory Warn‐on‐Forecast System (WoFS). The new LDA scheme first assigns a pseudo relative humidity between the cloud base and a specific layer based on the FED value. Then at each model layer, the pseudo relative humidity is converted to pseudo dewpoint temperature according to the corresponding air temperature. Some sensitivity experiments are performed to investigate how to assign and use GLM/FED in an optimum way. The impact of assimilating this pseudo dewpoint temperature on a short‐term severe weather forecast is preliminarily assessed in this proof‐of‐concept study. A high‐impact weather event in Kansas on 24 May 2021 is used to evaluate the performance of the new scheme on analyses and subsequent short‐term forecasts. The results show that the assimilation of additional FED‐based dewpoint temperature observations along with radar, satellite radiance, and cloud water can improve short‐term (3‐hr) forecast skill in terms of quantitative and qualitative verifications against the observations. The improvement is primarily due to the direct and indirect adjustment of dynamic and thermodynamic conditions through the LDA process. More specifically, the assimilation of FED‐based dewpoint temperature, in addition to the other observations currently used in WoFS, tends to enhance the ingredients required for thunderstorm formation, namely moisture, instability, and lifting mechanism.

     
    more » « less