We demonstrate thermodynamic profile estimation with data obtained using the MicroPulse DIAL such that the retrieval is entirely self contained. The only external input is surface meteorological variables obtained from a weather station installed on the instrument. The estimator provides products of temperature, absolute humidity and backscatter ratio such that cross dependencies between the lidar data products and raw observations are accounted for and the final products are self consistent. The method described here is applied to a combined oxygen DIAL, potassium HSRL, water vapor DIAL system operating at two pairs of wavelengths (nominally centered at 770 and 828 nm). We perform regularized maximum likelihood estimation through the Poisson Total Variation technique to suppress noise and improve the range of the observations. A comparison to 119 radiosondes indicates that this new processing method produces improved temperature retrievals, reducing total errors to less than 2 K below 3 km altitude and extending the maximum altitude of temperature retrievals to 5 km with less than 3 K error. The results of this work definitively demonstrates the potential for measuring temperature through the oxygen DIAL technique and furthermore that this can be accomplished with low-power semiconductor-based lidar sensors.
The precision of lidar measurements is limited by noise associated with the optical detection process. Photon noise also introduces biases in the second-order statistics of the data, such as the variances and fluxes of the measured temperature, wind, and species variations, and establishes noise floors in the computed fluctuation spectra. When the signal-to-noise ratio is low, these biases and noise floors can completely obscure the atmospheric processes being observed. We describe a novel data processing technique for eliminating the biases and noise floors. The technique involves acquiring two statistically independent datasets, covering the same altitude range and time period, from which the various second-order statistics are computed. The efficacy of the technique is demonstrated using Na Doppler lidar observations of temperature in the upper mesosphere and lower thermosphere acquired recently at McMurdo Station, Antarctica. The results show that this new technique enables observations of key atmospheric parameters in regions where the signal-to-noise ratio is far too low to apply conventional processing approaches.
more » « less- NSF-PAR ID:
- 10191865
- Publisher / Repository:
- Optical Society of America
- Date Published:
- Journal Name:
- Applied Optics
- Volume:
- 59
- Issue:
- 27
- ISSN:
- 1559-128X; APOPAI
- Format(s):
- Medium: X Size: Article No. 8259
- Size(s):
- Article No. 8259
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
-
more » « less
Abstract Random‐noise‐induced biases are inherent issues to the accurate derivation of second‐order statistical parameters (e.g., variances, fluxes, energy densities, and power spectra) from lidar and radar measurements. We demonstrate here for the first time an altitude‐interleaved method for eliminating such biases, following the original proposals by Gardner and Chu (2020,
https://doi.org/10.1364/ao.400375 ) who demonstrated a time‐interleaved method. Interleaving in altitude bins provides two statistically independent samples over the same time period and nearly the same altitude range, thus enabling the replacement of variances that include the noise‐induced biases with covariances that are intrinsically free of such biases. Comparing the interleaved method with previous variance subtraction (VS) and spectral proportion (SP) methods using gravity wave potential energy density calculated from Antarctic lidar data and from a forward model, this study finds the accuracy and precision of each method differing in various conditions, each with its own strengths and weakness. VS performs well in high‐SNR, yet its accuracy fails at lower‐SNR as it often yields negative values. SP is accurate and precise under high‐SNR, remaining accurate in worse conditions than VS would, yet develops a positive bias under low‐SNR. The interleaved method is accurate in all SNRs but requires a large number of samples to drive random‐noise terms in covariances toward zero and to compensate for the reduced precision due to the splitting of return signals. Therefore, selecting the proper bias removal/elimination method for actual signal and sample conditions is crucial in utilizing lidar/radar data, as neglecting this can conceal trends or overstate atmospheric variability. -
null (Ed.)Continuous advancements in LiDAR technology have enabled compelling wind turbulence measurements within the atmospheric boundary layer with range gates shorter than 20 m and sampling frequency of the order of 10 Hz. However, estimates of the radial velocity from the back-scattered laser beam are inevitably affected by an averaging process within each range gate, generally modeled as a convolution between the actual velocity projected along the LiDAR line-of-sight and a weighting function representing the energy distribution of the laser pulse along the range gate. As a result, the spectral energy of the turbulent velocity fluctuations is damped within the inertial sub-range with respective reduction of the velocity variance, and, thus, not allowing to take advantage of the achieved spatio-temporal resolution of the LiDAR technology. In this article, we propose to correct this turbulent energy damping on the LiDAR measurements by reversing the effect of a low-pass filter, which can be estimated directly from the LiDAR measurements. LiDAR data acquired from three different field campaigns are analyzed to describe the proposed technique, investigate the variability of the filter parameters and, for one dataset, assess the procedure for spectral LiDAR correction against sonic anemometer data. It is found that the order of the low-pass filter used for modeling the energy damping on the LiDAR velocity measurements has negligible effects on the correction of the second-order statistics of the wind velocity. In contrast, its cutoff frequency plays a significant role in the spectral correction encompassing the smoothing effects connected with the LiDAR gate length.more » « less
-
more » « less
This work presents the first demonstration of atmospheric temperature measurement using the differential absorption lidar (DIAL) technique. While DIAL is routinely used to measure atmospheric gases such as ozone and water vapor, almost no success has been found in using DIAL to measure atmospheric temperature. Attempts to measure temperature using a well-mixed gas like oxygen (
O 2) have largely failed based on a need for quantitative ancillary measurements of water vapor and atmospheric aerosols. Here, a lidar is described and demonstrated that simultaneously measuresO 2absorption, water vapor number density, and aerosol backscatter ratio. This combination of measurements allows for the first measurements of atmospheric temperature with useful accuracy. DIAL temperature measurements are presented to an altitude of 4km with 225m and 30min resolution with accuracy better than 3K . DIAL temperature data is compared to a co-located Raman lidar system and radiosondes to evaluate the system’s performance. Finally, an analysis of current performance characteristics is presented, which highlights pathways for future improvement of this proof-of-concept instrument. -
Since its development, the minimax framework has been one of the cornerstones of theoretical statistics, and has contributed to the popularity of many well-known estimators, such as the regularized M-estimators for high-dimensional problems. In this paper, we will first show through the example of sparse Gaussian sequence model, that the theoretical results under the classical minimax framework are insufficient for explaining empirical observations. In particular, both hard and soft thresholding estimators are (asymptotically) minimax, however, in practice they often exhibit sub-optimal performances at various signal-to-noise ratio (SNR) levels. The first contribution of this paper is to demonstrate that this issue can be resolved if the signal-to-noise ratio is taken into account in the construction of the parameter space. We call the resulting minimax framework the signal-to-noise ratio aware minimaxity. The second contribution of this paper is to showcase how one can use higher-order asymptotics to obtain accurate approximations of the SNR-aware minimax risk and discover minimax estimators. The theoretical findings obtained from this refined minimax framework provide new insights and practical guidance for the estimation of sparse signals.more » « less
