An official website of the United States government
Abstract. Remote sensing measurements have been widely used to estimate the planetary boundary layer height (PBLHT). Each remote sensing approach offers unique strengths and faces different limitations. In this study, we use machine learning (ML) methods to produce a best-estimate PBLHT (PBLHT-BE-ML) by integrating four PBLHT estimates derived from remote sensing measurements at the Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) observatory. Three ML models – random forest (RF) classifier, RF regressor, and light gradient-boosting machine (LightGBM) – were trained on a dataset from 2017 to 2023 that included radiosonde, various remote sensing PBLHT estimates, and atmospheric meteorological conditions. Evaluations indicated that PBLHT-BE-ML from all three models improved alignment with the PBLHT derived from radiosonde data (PBLHT-SONDE), with LightGBM demonstrating the highest accuracy under both stable and unstable boundary layer conditions. Feature analysis revealed that the most influential input features at the SGP site were the PBLHT estimates derived from (a) potential temperature profiles retrieved using Raman lidar (RL) and atmospheric emitted radiance interferometer (AERI) measurements (PBLHT-THERMO), (b) vertical velocity variance profiles from Doppler lidar (PBLHT-DL), and (c) aerosol backscatter profiles from micropulse lidar (PBLHT-MPL). The trained models were then used to predict PBLHT-BE-ML at a temporal resolution of 10 min, effectively capturing the diurnal evolution of PBLHT and its significant seasonal variations, with the largest diurnal variation observed over summer at the SGP site. We applied these trained models to data from the ARM Eastern Pacific Cloud Aerosol Precipitation Experiment (EPCAPE) field campaign (EPC), where the PBLHT-BE-ML, particularly with the LightGBM model, demonstrated improved accuracy against PBLHT-SONDE. Analyses of model performance at both the SGP and EPC sites suggest that expanding the training dataset to include various surface types, such as ocean and ice-covered areas, could further enhance ML model performance for PBLHT estimation across varied geographic regions.
more »
« less
ARMing the Edge: Designing Edge Computing–Capable Machine Learning Algorithms to Target ARM Doppler Lidar Processing
Abstract There is a need for long-term observations of cloud and precipitation fall speeds in validating and improving rainfall forecasts from climate models. To this end, the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility Southern Great Plains (SGP) site at Lamont, Oklahoma, hosts five ARM Doppler lidars that can measure cloud and aerosol properties. In particular, the ARM Doppler lidars record Doppler spectra that contain information about the fall speeds of cloud and precipitation particles. However, due to bandwidth and storage constraints, the Doppler spectra are not routinely stored. This calls for the automation of cloud and rain detection in ARM Doppler lidar data so that the spectral data in clouds can be selectively saved and further analyzed. During the ARMing the Edge field experiment, a Waggle node capable of performing machine learning applications in situ was deployed at the ARM SGP site for this purpose. In this paper, we develop and test four algorithms for the Waggle node to automatically classify ARM Doppler lidar data. We demonstrate that supervised learning using a ResNet50-based classifier will classify 97.6% of the clear-air images and 94.7% of cloudy images correctly, outperforming traditional peak detection methods. We also show that a convolutional autoencoder paired withk-means clustering identifies 10 clusters in the ARM Doppler lidar data. Three clusters correspond to mostly clear conditions with scattered high clouds, and seven others correspond to cloudy conditions with varying cloud-base heights.
more »
« less
- Award ID(s):
- 1935984
- PAR ID:
- 10463110
- Publisher / Repository:
- American Meteorological Society
- Date Published:
- Journal Name:
- Artificial Intelligence for the Earth Systems
- Volume:
- 2
- Issue:
- 4
- ISSN:
- 2769-7525
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract. Accurate measurement of the mixed layer height (MLH) is a key observational capability necessary for many studies in weather forecasting, air quality assessment, and surface-atmosphere exchange. However, continuous MLH monitoring with backscatter lidars remains challenging under complex atmospheric conditions, including cloudy conditions and in the presence of residual layers. This study evaluates two complementary MLH retrieval algorithms using a single MicroPulse Differential Absorption Lidar (MPD): an aerosol-based approach that analyzes aerosol backscatter gradients with a wavelet technique and a thermodynamic technique based on the vertical structure of virtual potential temperature profiles. Both techniques were compared against MLH estimates from radiosondes, a Doppler wind lidar, and a high-resolution weather model using data from the M2HATS field campaign in Tonopah, NV, USA, supplemented by a smaller dataset from Boulder, CO, USA. The aerosol method achieved high temporal resolution and agreement with radiosonde MLH estimates under convective conditions (R2= 0.819–0.919), but its MLH estimates deviated from other methods during morning and evening transitions due to residual layer interference. The thermodynamic method avoided these problems but had coarser resolution and degraded instrument performance beneath clouds (R2= 0.661–0.845). Because lidar generally cannot penetrate clouds, conditions with clouds at or below the MLH are not considered, while those with clouds above the MLH are retained. The study highlights the strengths and weaknesses of each method. Together, they offer a path toward more reliable automatic MLH monitoring with a single instrument by capturing when different MLH definitions converge.more » « less
-
Abstract. The states of coupling between clouds andsurface or boundary layer have been investigated much more extensively formarine stratocumulus clouds than for continental low clouds, partly due tomore complex thermodynamic structures over land. A manifestation is a lackof robust remote sensing methods to identify coupled and decoupled cloudsover land. Following the idea for determining cloud coupling over the ocean,we have generalized the concept of coupling and decoupling to low cloudsover land, based on potential temperature profiles. Furthermore, by usingample measurements from lidar and a suite of surface meteorologicalinstruments at the U.S. Department of Energy's Atmospheric RadiationMeasurement Program's Southern Great Plains site from 1998 to 2019, we havedeveloped a method to simultaneously retrieve the planetary boundary layer(PBL) height (PBLH) and coupled states under cloudy conditions during thedaytime. The new lidar-based method relies on the PBLH, the liftedcondensation level, and the cloud base to diagnose the cloud coupling. Thecoupled states derived from this method are highly consistent with thosederived from radiosondes. Retrieving the PBLH under cloudy conditions, whichhas been a persistent problem in lidar remote sensing, is resolved in thisstudy. Our method can lead to high-quality retrievals of the PBLH undercloudy conditions and the determination of cloud coupling states. With thenew method, we find that coupled clouds are sensitive to changes in the PBLwith a strong diurnal cycle, whereas decoupled clouds and the PBL are weaklyrelated. Since coupled and decoupled clouds have distinct features, our newmethod offers an advanced tool to separately investigate them in climatesystems.more » « less
-
LiDAR (Light Detection And Ranging) is an indispensable sensor for precise long- and wide-range 3D sensing, which directly benefited the recent rapid deployment of autonomous driving (AD). Meanwhile, such a safety-critical application strongly motivates its security research. A recent line of research demonstrates that one can manipulate the LiDAR point cloud and fool object detection by firing malicious lasers against LiDAR. However, these efforts evaluate only a specific LiDAR (VLP-16) and do not consider the state-of-the-art defense mechanisms in the recent LiDARs, so-called next-generation LiDARs. In this WIP work, we report our recent progress in the security analysis of the next-generation LiDARs. We identify a new type of LiDAR spoofing attack applicable to a much more general and recent set of LiDARs. We find that our attack can remove >72% of points in a 10×10 m2 area and can remove real vehicles in the physical world. We also discuss our future plans.more » « less
-
Abstract A field experiment was conducted to investigate the effects of the thrust force induced by utility-scale wind turbines on the incoming wind field. Five wind profiling LiDARs and a scanning Doppler pulsed wind LiDAR were deployed in the proximity of a row of four wind turbines located over relatively flat terrain, both before and after the construction of the wind farm. The analysis of the LiDAR data collected during the pre-construction phase enables quantifying the wind map of the site, which is then leveraged to correct the post-construction LiDAR data and isolate rotor-induced effects on the incoming wind field. The analysis of the profiling LiDAR data allows for the identification of the induction zone upstream of the turbine rotors, with an increasing velocity deficit moving from the top tip towards the bottom tip of the rotor. The largest wind speed reduction (about 5%) is observed for convective conditions and incoming hub-height wind speed between cut-in and rated wind speeds. The scanning LiDAR data indicate the presence of speedup regions within the gaps between adjacent turbine rotors. Speedup increases with reducing the transverse distance between the rotors, atmospheric instability (maximum 15%), while a longer streamwise extent of the speedup region is observed under stable atmospheric conditions.more » « less
