An official website of the United States government
Abstract Observational data inherently contain noise which manifests as uncertainties in the measured parameters and creates positive biases or noise floors in second‐order products like variances, fluxes, and spectra. Historical methods estimate and subsequently subtract noise floors, but struggle with accuracy. Gardner and Chu (2020,doi.org/10.1364/AO.400375) proposed an interleaved data processing method, which inherently eliminates biases from variances and fluxes, and suggested that the method could also eliminate noise floors of power spectra. We investigate the interleaved method for spectral analysis of atmospheric waves through theoretical studies, forward modeling, and demonstration with lidar data. Our work shows that calculating the cross‐power spectral density (CPSD) from two interleaved subsamples does reduce the spectral noise floor significantly. However, only the Co‐PSD (the real part of CPSD) eliminates the noise floor completely, while taking the absolute magnitude of CPSD adds a reduced noise floor back to the spectrum when the sample number is finite. This reduced noise floor can be further minimized through averaging over more observations, completely different from traditional spectrum calculations whose noise floor cannot be reduced by incorporating more samples. We demonstrate the first application of the interleaved method to spectral data, successfully eliminating the noise floor using the Co‐PSD in a forward model and in lidar observations of the vertical wavenumber of gravity waves at McMurdo, Antarctica. This high accuracy is gained by sacrificing precision due to photon‐count splitting, requiring additional observations to counter this effect. We provide quantitative assessment of accuracy and precision as well as application recommendations.
more »
« less
Comparison of Three Methodologies for Removal of Random‐Noise‐Induced Biases From Second‐Order Statistical Parameters of Lidar and Radar Measurements
Abstract Random‐noise‐induced biases are inherent issues to the accurate derivation of second‐order statistical parameters (e.g., variances, fluxes, energy densities, and power spectra) from lidar and radar measurements. We demonstrate here for the first time an altitude‐interleaved method for eliminating such biases, following the original proposals by Gardner and Chu (2020,https://doi.org/10.1364/ao.400375) who demonstrated a time‐interleaved method. Interleaving in altitude bins provides two statistically independent samples over the same time period and nearly the same altitude range, thus enabling the replacement of variances that include the noise‐induced biases with covariances that are intrinsically free of such biases. Comparing the interleaved method with previous variance subtraction (VS) and spectral proportion (SP) methods using gravity wave potential energy density calculated from Antarctic lidar data and from a forward model, this study finds the accuracy and precision of each method differing in various conditions, each with its own strengths and weakness. VS performs well in high‐SNR, yet its accuracy fails at lower‐SNR as it often yields negative values. SP is accurate and precise under high‐SNR, remaining accurate in worse conditions than VS would, yet develops a positive bias under low‐SNR. The interleaved method is accurate in all SNRs but requires a large number of samples to drive random‐noise terms in covariances toward zero and to compensate for the reduced precision due to the splitting of return signals. Therefore, selecting the proper bias removal/elimination method for actual signal and sample conditions is crucial in utilizing lidar/radar data, as neglecting this can conceal trends or overstate atmospheric variability.
more »
« less
- PAR ID:
- 10374452
- Publisher / Repository:
- DOI PREFIX: 10.1029
- Date Published:
- Journal Name:
- Earth and Space Science
- Volume:
- 9
- Issue:
- 1
- ISSN:
- 2333-5084
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract A very high spatial resolution (∼25 m pixel at 90 km altitude) OH airglow imager was installed at the Andes Lidar Observatory on Cerro Pachón, Chile, in February 2016. This instrument was collocated with a Na wind‐temperature lidar. On 1 March 2016, the lidar data showed that the atmosphere was dynamically unstable before 0100 UT and thus conducive to the formation of Kelvin‐Helmholtz instabilities (KHIs). The imager revealed the presence of a KHI and an apparent atmospheric gravity wave (AGW) propagating approximately perpendicular to the plane of primary KHI motions. The AGW appears to have induced modulations of the shear layer leading to misalignments of the emerging KHI billows. These enabled strong KHI billow interactions, as they achieved large amplitudes and a rapid transition to turbulence thereafter. The interactions manifested themselves as vortex tube and knot features that were earlier identified in laboratory studies, as discussed in Thorpe (1987,https://doi.org/10.1029/JC092iC05p05231; 2002,https://doi.org/10.1002/qj.200212858307) and inferred to be widespread in the atmosphere based on features seen in tropospheric clouds but which have never been identified in previous upper atmospheric observations. This study presents the first high‐resolution airglow imaging observation of these KHI interaction dynamics that drive rapid transitions to turbulence and suggest the potential importance of these dynamics in the mesosphere and at other altitudes. A companion paper (Fritts et al., 2020,https://doi.org/10.1029/2020JD033412) modeling these dynamics confirms that the vortex tubes and knots yield more rapid and significantly enhanced turbulence relative to the internal instabilities of individual KHI billows.more » « less
-
Abstract We introduce a new framework called Machine Learning (ML) based Auroral Ionospheric electrodynamics Model (ML‐AIM). ML‐AIM solves a current continuity equation by utilizing the ML model of Field Aligned Currents of Kunduri et al. (2020,https://doi.org/10.1029/2020JA027908), the FAC‐derived auroral conductance model of Robinson et al. (2020,https://doi.org/10.1029/2020JA028008), and the solar irradiance conductance model of Moen and Brekke (1993,https://doi.org/10.1029/92gl02109). The ML‐AIM inputs are 60‐min time histories of solar wind plasma, interplanetary magnetic fields (IMF), and geomagnetic indices, and its outputs are ionospheric electric potential, electric fields, Pedersen/Hall currents, and Joule Heating. We conduct two ML‐AIM simulations for a weak geomagnetic activity interval on 14 May 2013 and a geomagnetic storm on 7–8 September 2017. ML‐AIM produces physically accurate ionospheric potential patterns such as the two‐cell convection pattern and the enhancement of electric potentials during active times. The cross polar cap potentials (ΦPC) from ML‐AIM, the Weimer (2005,https://doi.org/10.1029/2004ja010884) model, and the Super Dual Auroral Radar Network (SuperDARN) data‐assimilated potentials, are compared to the ones from 3204 polar crossings of the Defense Meteorological Satellite Program F17 satellite, showing better performance of ML‐AIM than others. ML‐AIM is unique and innovative because it predicts ionospheric responses to the time‐varying solar wind and geomagnetic conditions, while the other traditional empirical models like Weimer (2005,https://doi.org/10.1029/2004ja010884) designed to provide a quasi‐static ionospheric condition under quasi‐steady solar wind/IMF conditions. Plans are underway to improve ML‐AIM performance by including a fully ML network of models of aurora precipitation and ionospheric conductance, targeting its characterization of geomagnetically active times.more » « less
-
Abstract Both high‐power large aperture radars and smaller meteor radars readily observe the dense head plasma produced as a meteoroid ablates. However, determining the mass of such meteors based on the information returned by the radar is challenging. We present a new method for deriving meteor masses from single‐frequency radar measurements, using a physics‐based plasma model and finite‐difference time‐domain (FDTD) simulations. The head plasma model derived in Dimant and Oppenheim (2017),https://doi.org/10.1002/2017ja023963depends on the meteoroids altitude, speed, and size. We use FDTD simulations of a radar pulse interacting with such head plasmas to determine the radar cross section (RCS) that a radar system would observe for a meteor with a given set of physical properties. By performing simulations over the observed parameter space, we construct tables relating meteor size, velocity, and altitude to RCS. We then use these tables to map a set of observations from the MAARSY radar (53.5 MHz) to fully defined plasma distributions, from which masses are calculated. To validate these results, we repeat the analysis using observations of the same meteors by the EISCAT radar (929 MHz). The resulting masses are strongly linearly correlated; however, the masses derived from EISCAT measurements are on average 1.33 times larger than those derived from MAARSY measurements. Since this method does not require dual‐frequency measurements for mass determination, only validation, it can be applied in the future to observations made by many single‐frequency radar systems.more » « less
-
Abstract Following the reanalysis of individual experimental runs of some widely cited studies (Jain et al., 2018,https://doi.org/10.1002/2017JB014847), we revisit the global data analysis of Korenaga and Karato (2008,https://doi.org/10.1029/2007JB005100) with a significantly improved version of their Markov chain Monte Carlo inversion. Their algorithm, previously corrected by Mullet et al. () to minimize potential parameter bias, is further modified here to estimate more efficiently interrun biases in global data sets. Using the refined Markov chain Monte Carlo inversion technique, we simultaneously analyze experimental data on the deformation of olivine aggregates compiled from different studies. Realistic composite rheological models, including both diffusion and dislocation creep, are adopted, and the role of dislocation‐accommodated grain boundary sliding is also investigated. Furthermore, the influence of interrun biases on inversion results is studied using experimental and synthetic data. Our analysis shows that existing data can tightly constrain the grain‐size exponent for diffusion creep at ∼2, which is different from the value commonly assumed (p= 3). Different data sets and model assumptions, however, yield nonoverlapping estimates on other flow‐law parameters, and the flow‐law parameters for grain boundary sliding are poorly resolved in most cases. We thus provide a few plausible candidate flow‐law models for olivine rheology to facilitate future geodynamic modeling. The availability of more data that explore a wider range of experimental conditions, especially higher pressures, is essential to improve our understanding of upper mantle rheology.more » « less
