skip to main content


Title: Comparison of Three Methodologies for Removal of Random‐Noise‐Induced Biases From Second‐Order Statistical Parameters of Lidar and Radar Measurements
Abstract

Random‐noise‐induced biases are inherent issues to the accurate derivation of second‐order statistical parameters (e.g., variances, fluxes, energy densities, and power spectra) from lidar and radar measurements. We demonstrate here for the first time an altitude‐interleaved method for eliminating such biases, following the original proposals by Gardner and Chu (2020,https://doi.org/10.1364/ao.400375) who demonstrated a time‐interleaved method. Interleaving in altitude bins provides two statistically independent samples over the same time period and nearly the same altitude range, thus enabling the replacement of variances that include the noise‐induced biases with covariances that are intrinsically free of such biases. Comparing the interleaved method with previous variance subtraction (VS) and spectral proportion (SP) methods using gravity wave potential energy density calculated from Antarctic lidar data and from a forward model, this study finds the accuracy and precision of each method differing in various conditions, each with its own strengths and weakness. VS performs well in high‐SNR, yet its accuracy fails at lower‐SNR as it often yields negative values. SP is accurate and precise under high‐SNR, remaining accurate in worse conditions than VS would, yet develops a positive bias under low‐SNR. The interleaved method is accurate in all SNRs but requires a large number of samples to drive random‐noise terms in covariances toward zero and to compensate for the reduced precision due to the splitting of return signals. Therefore, selecting the proper bias removal/elimination method for actual signal and sample conditions is crucial in utilizing lidar/radar data, as neglecting this can conceal trends or overstate atmospheric variability.

 
more » « less
Award ID(s):
2110428 2029162
NSF-PAR ID:
10374452
Author(s) / Creator(s):
 ;  
Publisher / Repository:
DOI PREFIX: 10.1029
Date Published:
Journal Name:
Earth and Space Science
Volume:
9
Issue:
1
ISSN:
2333-5084
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    A very high spatial resolution (25 m pixel at 90 km altitude) OH airglow imager was installed at the Andes Lidar Observatory on Cerro Pachón, Chile, in February 2016. This instrument was collocated with a Na wind‐temperature lidar. On 1 March 2016, the lidar data showed that the atmosphere was dynamically unstable before 0100 UT and thus conducive to the formation of Kelvin‐Helmholtz instabilities (KHIs). The imager revealed the presence of a KHI and an apparent atmospheric gravity wave (AGW) propagating approximately perpendicular to the plane of primary KHI motions. The AGW appears to have induced modulations of the shear layer leading to misalignments of the emerging KHI billows. These enabled strong KHI billow interactions, as they achieved large amplitudes and a rapid transition to turbulence thereafter. The interactions manifested themselves as vortex tube and knot features that were earlier identified in laboratory studies, as discussed in Thorpe (1987,https://doi.org/10.1029/JC092iC05p05231; 2002,https://doi.org/10.1002/qj.200212858307) and inferred to be widespread in the atmosphere based on features seen in tropospheric clouds but which have never been identified in previous upper atmospheric observations. This study presents the first high‐resolution airglow imaging observation of these KHI interaction dynamics that drive rapid transitions to turbulence and suggest the potential importance of these dynamics in the mesosphere and at other altitudes. A companion paper (Fritts et al., 2020,https://doi.org/10.1029/2020JD033412) modeling these dynamics confirms that the vortex tubes and knots yield more rapid and significantly enhanced turbulence relative to the internal instabilities of individual KHI billows.

     
    more » « less
  2. Abstract

    We introduce a new framework called Machine Learning (ML) based Auroral Ionospheric electrodynamics Model (ML‐AIM). ML‐AIM solves a current continuity equation by utilizing the ML model of Field Aligned Currents of Kunduri et al. (2020,https://doi.org/10.1029/2020JA027908), the FAC‐derived auroral conductance model of Robinson et al. (2020,https://doi.org/10.1029/2020JA028008), and the solar irradiance conductance model of Moen and Brekke (1993,https://doi.org/10.1029/92gl02109). The ML‐AIM inputs are 60‐min time histories of solar wind plasma, interplanetary magnetic fields (IMF), and geomagnetic indices, and its outputs are ionospheric electric potential, electric fields, Pedersen/Hall currents, and Joule Heating. We conduct two ML‐AIM simulations for a weak geomagnetic activity interval on 14 May 2013 and a geomagnetic storm on 7–8 September 2017. ML‐AIM produces physically accurate ionospheric potential patterns such as the two‐cell convection pattern and the enhancement of electric potentials during active times. The cross polar cap potentials (ΦPC) from ML‐AIM, the Weimer (2005,https://doi.org/10.1029/2004ja010884) model, and the Super Dual Auroral Radar Network (SuperDARN) data‐assimilated potentials, are compared to the ones from 3204 polar crossings of the Defense Meteorological Satellite Program F17 satellite, showing better performance of ML‐AIM than others. ML‐AIM is unique and innovative because it predicts ionospheric responses to the time‐varying solar wind and geomagnetic conditions, while the other traditional empirical models like Weimer (2005,https://doi.org/10.1029/2004ja010884) designed to provide a quasi‐static ionospheric condition under quasi‐steady solar wind/IMF conditions. Plans are underway to improve ML‐AIM performance by including a fully ML network of models of aurora precipitation and ionospheric conductance, targeting its characterization of geomagnetically active times.

     
    more » « less
  3. Abstract

    Both high‐power large aperture radars and smaller meteor radars readily observe the dense head plasma produced as a meteoroid ablates. However, determining the mass of such meteors based on the information returned by the radar is challenging. We present a new method for deriving meteor masses from single‐frequency radar measurements, using a physics‐based plasma model and finite‐difference time‐domain (FDTD) simulations. The head plasma model derived in Dimant and Oppenheim (2017),https://doi.org/10.1002/2017ja023963depends on the meteoroids altitude, speed, and size. We use FDTD simulations of a radar pulse interacting with such head plasmas to determine the radar cross section (RCS) that a radar system would observe for a meteor with a given set of physical properties. By performing simulations over the observed parameter space, we construct tables relating meteor size, velocity, and altitude to RCS. We then use these tables to map a set of observations from the MAARSY radar (53.5 MHz) to fully defined plasma distributions, from which masses are calculated. To validate these results, we repeat the analysis using observations of the same meteors by the EISCAT radar (929 MHz). The resulting masses are strongly linearly correlated; however, the masses derived from EISCAT measurements are on average 1.33 times larger than those derived from MAARSY measurements. Since this method does not require dual‐frequency measurements for mass determination, only validation, it can be applied in the future to observations made by many single‐frequency radar systems.

     
    more » « less
  4. Abstract

    Narrowband stimulated electromagnetic emissions (NSEE), a component of radio emissions created during high‐frequency (HF) radiowave ionospheric modification experiments, occur within 1 kHz of the HF pump frequency. NSEE was observed and studied for the first time at the HAARP (High Frequency Active Auroral Research Program) facility in Alaska (Norin et al., 2009,https://doi.org/10.1103/PhysRevLett.102.065003; Bernhardt et al., 2010,https://doi.org/10.1103/PhysRevLett.104.165004). Magnetized stimulated Brillouin scatter (MSBS) is a component of NSEE, which was also first observed at HAARP (Norin et al., 2009,https://doi.org/10.1103/PhysRevLett.102.065003; Bernhardt et al., 2010,https://doi.org/10.1103/PhysRevLett.104.165004) and later at European Incoherent Scatter Scientific Association (Fu et al., 2015,https://doi.org/10.5194/angeo-33-983-2015). Ion‐acoustic and electrostatic ion cyclotron modes are the daughter products of MSBS and can be used for the determination of the electron temperature and of the presence of minor ion species, respectively, in the HF‐modified ionosphere. Here we present the first observations of the MSBS process at magnetic midlatitudes, excited during radio wave ionospheric modification experiments at the Arecibo Observatory. The NSEE observations, in combination with a theoretical model and the wave matching conditions, are used to estimate background ionospheric parameters. A qualitative comparison of the MSBS component of the NSEE spectrum with the thermal ion line measured by incoherent scatter radar is presented.

     
    more » « less
  5. Abstract

    We present empirical conductance relations that are derived from incoherent scatter radar observations and correlated with all sky imager observations to identify the morphology of the aurora. We use 75,461 events collected using the Poker Flat Incoherent Scatter Radar (PFISR) with associated all sky imagers observations spanning the years 2012–2016. In addition to classifying these events based on auroral morphology, we estimated the Hall and Pedersen conductance and the differential number flux from which the energy flux and the average energy can be calculated. The differential number flux was estimated using the maximum entropy inversion method described in Semeter and Kamalabadi (2005,https://doi.org/10.1029/2004RS003042), but now incorporating the Fang et al. (2010,https://doi.org/10.1029/2010GL045406) ionization model. The main results of this investigation are the power law equations that describe the median, 90th, and 10th percentile Hall and Pedersen conductance as a function of energy flux and average energy. These power law fits are performed for different auroral morphology including all events, discrete, diffuse, and pulsating auroral events. The median Pedersen conductance is found to be in good agreement with past empirical conductance specifications by Robinson et al. (1987,https://doi.org/10.1029/JA092iA03p02565); however, the median Hall conductance from the PFISR observations is found to be larger than the empirical Hall conductance formulas by Robinson et al. (1987,https://doi.org/10.1029/JA092iA03p02565). Pulsating aurora is found to be the most frequently occurring auroral morphology. Furthermore, pulsating aurora has an important contribution to Hall conductance since it has higher average energies than discrete aurora. The results from this investigation are applicable to space weather models and may enable better agreement between model‐data comparisons.

     
    more » « less