skip to main content

Explore Research Products in the NSF-PAR

Title: Enabling efficient execution of a variational data assimilation application

Remote sensing observational instruments are critical for better understanding and predicting severe weather. Observational data from such instruments, such as Doppler radar data, for example, are often processed for assimilation into numerical weather prediction models. As such instruments become more sophisticated, the amount of data to be processed grows and requires efficient variational analysis tools. Here we examine the code that implements the popular SAMURAI (Spline Analysis at Mesoscale Utilizing Radar and Aircraft Instrumentation) technique for estimating the atmospheric state for a given set of observations. We employ a number of techniques to significantly improve the code’s performance, including porting it to run on standard HPC clusters, analyzing and optimizing its single-node performance, implementing a more efficient nonlinear optimization method, and enabling the use of GPUs via OpenACC. Our efforts thus far have yielded more than 100x improvement over the original code on large test problems of interest to the community.

  more » « less
Award ID(s):
2103776
NSF-PAR ID:
10370569
Author(s) / Creator(s):
 ;  ;  ;  ;  ;  ;  
Publisher / Repository:
SAGE Publications
Date Published:
Journal Name:
The International Journal of High Performance Computing Applications
ISSN:
1094-3420
Page Range / eLocation ID:
Article No. 109434202211198
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ( , Artificial Intelligence for the Earth Systems)
    Abstract

    The quantification of storm updrafts remains unavailable for operational forecasting despite their inherent importance to convection and its associated severe weather hazards. Updraft proxies, like overshooting top area from satellite images, have been linked to severe weather hazards but only relate to a limited portion of the total storm updraft. This study investigates if a machine learning model, namely, U-Nets, can skillfully retrieve maximum vertical velocity and its areal extent from three-dimensional gridded radar reflectivity alone. The machine learning model is trained using simulated radar reflectivity and vertical velocity from the National Severe Storm Laboratory’s convection permitting Warn-on-Forecast System (WoFS). A parametric regression technique using the sinh–arcsinh–normal distribution is adapted to run with U-Nets, allowing for both deterministic and probabilistic predictions of maximum vertical velocity. The best models after hyperparameter search provided less than 50% root mean squared error, a coefficient of determination greater than 0.65, and an intersection over union (IoU) of more than 0.45 on the independent test set composed of WoFS data. Beyond the WoFS analysis, a case study was conducted using real radar data and corresponding dual-Doppler analyses of vertical velocity within a supercell. The U-Net consistently underestimates the dual-Doppler updraft speed estimates by 50%. Meanwhile, the area of the 5 and 10 m s−1updraft cores shows an IoU of 0.25. While the above statistics are not exceptional, the machine learning model enables quick distillation of 3D radar data that is related to the maximum vertical velocity, which could be useful in assessing a storm’s severe potential.

    Significance Statement

    All convective storm hazards (tornadoes, hail, heavy rain, straight line winds) can be related to a storm’s updraft. Yet, there is no direct measurement of updraft speed or area available for forecasters to make their warning decisions from. This paper addresses the lack of observational data by providing a machine learning solution that skillfully estimates the maximum updraft speed within storms from only the radar reflectivity 3D structure. After further vetting the machine learning solutions on additional real-world examples, the estimated storm updrafts will hopefully provide forecasters with an added tool to help diagnose a storm’s hazard potential more accurately.

     
    more » « less
  2. ; ; ; ( , Journal of Atmospheric and Oceanic Technology)
    Abstract

    While land-based high-frequency (HF) radars are the only instruments capable of resolving both the temporal and spatial variability of surface currents in the coastal ocean, recent high-resolution views suggest that the coastal ocean is more complex than presently deployed radar systems are able to reveal. This work uses a hybrid system, having elements of both phased arrays and direction finding radars, to improve the azimuthal resolution of HF radars. Data from two radars deployed along the U.S. East Coast and configured as 8-antenna grid arrays were used to evaluate potential direction finding and signal, or emitter, detection methods. Direction finding methods such as maximum likelihood estimation generally performed better than the well-known multiple signal classification (MUSIC) method given identical emitter detection methods. However, accurately estimating the number of emitters present in HF radar observations is a challenge. As MUSIC’s direction-of-arrival (DOA) function permits simple empirical tests that dramatically aid the detection process, MUSIC was found to be the superior method in this study. The 8-antenna arrays were able to provide more accurate estimates of MUSIC’s noise subspace than typical 3-antenna systems, eliminating the need for a series of empirical parameters to control MUSIC’s performance. Code developed for this research has been made available in an online repository.

     
    more » « less
  3. ; ; ; ; ; ( , Weather and Forecasting)
    Abstract

    Assimilating radar reflectivity into convective-scale NWP models remains a challenging topic in radar data assimilation. A primary reason is that the reflectivity forward observation operator is highly nonlinear. To address this challenge, a power transformation function is applied to the WRF Model’s hydrometeor and water vapor mixing ratio variables in this study. Three 3D variational data assimilation experiments are performed and compared for five high-impact weather events that occurred in 2019: (i) a control experiment that assimilates reflectivity using the original hydrometeor mixing ratios as control variables, (ii) an experiment that assimilates reflectivity using power-transformed hydrometeor mixing ratios as control variables, and (iii) an experiment that assimilates reflectivity and retrieved pseudo–water vapor observations using power-transformed hydrometeor and water vapor mixing ratios (qυ) as control variables. Both qualitative and quantitative evaluations are performed for 0–3-h forecasts from the five cases. The analysis and forecast performance in the two experiments with power-transformed mixing ratios is better than the control experiment. Notably, the assimilation of pseudo–water vapor with power-transformedqυas an additional control variable is found to improve the performance of the analysis and short-term forecasts for all cases. In addition, the convergence rate of the cost function minimization for the two experiments that use the power transformation is faster than that of the control experiments.

    Significance Statement

    The effective use of radar reflectivity observations in any data assimilation scheme remains an important research topic because reflectivity observations explicitly include information about hydrometeors and also implicitly include information about the distribution of moisture within storms. However, it is difficult to assimilate reflectivity because the reflectivity forward observation operator is highly nonlinear. This study seeks to identify a more effective way to assimilate reflectivity into a convective-scale NWP model to improve the accuracy of predictions of high-impact weather events.

     
    more » « less
  4. ; ; ; ( , IEEE Conference on Computer Vision and Pattern Recognition)
    null (Ed.)
    Vehicle detection with visual sensors like lidar and camera is one of the critical functions enabling autonomous driving. While they generate fine-grained point clouds or high-resolution images with rich information in good weather conditions, they fail in adverse weather (e.g., fog) where opaque particles distort lights and significantly reduce visibility. Thus, existing methods relying on lidar or camera experience significant performance degradation in rare but critical adverse weather conditions. To remedy this, we resort to exploiting complementary radar, which is less impacted by adverse weather and becomes prevalent on vehicles. In this paper, we present Multimodal Vehicle Detection Network (MVDNet), a two-stage deep fusion detector, which first generates proposals from two sensors and then fuses region-wise features between multimodal sensor streams to improve final detection results. To evaluate MVDNet, we create a procedurally generated training dataset based on the collected raw lidar and radar signals from the open-source Oxford Radar Robotcar. We show that the proposed MVDNet surpasses other state-of-the-art methods, notably in terms of Average Precision (AP), especially in adverse weather conditions. The code and data are available at https://github.com/qiank10/MVDNet. 
    more » « less
  5. ; ; ; ; ( , Remote Sensing)

    We present a machine-learning approach to detect and analyze meteor echoes (MADAME), which is a radar data processing workflow featuring advanced machine-learning techniques using both supervised and unsupervised learning. Our results demonstrate that YOLOv4, a convolutional neural network (CNN)-based one-stage object detection model, performs remarkably well in detecting and identifying meteor head and trail echoes within processed radar signals. The detector can identify more than 80 echoes per minute in the testing data obtained from the Jicamarca high power large aperture (HPLA) radar. MADAME is also capable of autonomously processing data in an interferometer mode, as well as determining the target’s radiant source and vector velocity. In the testing data, the Eta Aquarids meteor shower could be clearly identified from the meteor radiant source distribution analyzed automatically by MADAME, thereby demonstrating the proposed algorithm’s functionality. In addition, MADAME found that about 50 percent of the meteors were traveling in inclined and near-inclined circular orbits. Furthermore, meteor head echoes with a trail are more likely to originate from shower meteor sources. Our results highlight the capability of advanced machine-learning techniques in radar signal processing, providing an efficient and powerful tool to facilitate future and new meteor research.

     
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email