An official website of the United States government
Data visualizations can reveal trends and patterns that are not otherwise obvious from the raw data or summary statistics. While visualizing low-dimensional data is relatively straightforward (for example, plotting the change in a variable over time as (x,y) coordinates on a graph), it is not always obvious how to visualize high-dimensional datasets in a similarly intuitive way. Here we present HypeTools, a Python toolbox for visualizing and manipulating large, high-dimensional datasets. Our primary approach is to use dimensionality reduction techniques (Pearson, 1901; Tipping & Bishop, 1999) to embed high-dimensional datasets in a lower-dimensional space, and plot the data using a simple (yet powerful) API with many options for data manipulation [e.g. hyperalignment (Haxby et al., 2011), clustering, normalizing, etc.] and plot styling. The toolbox is designed around the notion of data trajectories and point clouds. Just as the position of an object moving through space can be visualized as a 3D trajectory, HyperTools uses dimensionality reduction algorithms to create similar 2D and 3D trajectories for time series of high-dimensional observations. The trajectories may be plotted as interactive static plots or visualized as animations. These same dimensionality reduction and alignment algorithms can also reveal structure in static datasets (e.g. collections of observations or attributes). We present several examples showcasing how using our toolbox to explore data through trajectories and low-dimensional embeddings can reveal deep insights into datasets across a wide variety of domains.
more »
« less
pyDARN: A Python software for visualizing SuperDARN radar data
The Super Dual Auroral Radar Network (SuperDARN) is an international network of high frequency coherent scatter radars that are used for monitoring the electrodynamics of the Earth’s upper atmosphere at middle, high, and polar latitudes in both hemispheres. pyDARN is an open-source Python-based library developed specifically for visualizing SuperDARN radar data products. It provides various plotting functions of different types of SuperDARN data, including time series plot, range-time parameter plot, fields of view, full scan, and global convection map plots. In this paper, we review the different types of SuperDARN data products, pyDARN’s development history and goals, the current implementation of pyDARN, and various plotting and analysis functionalities. We also discuss applications of pyDARN, how it can be combined with other existing Python software for scientific analysis, challenges for pyDARN development and future plans. Examples showing how to read, visualize, and interpret different SuperDARN data products using pyDARN are provided as a Jupyter notebook.
more »
« less
- PAR ID:
- 10412899
- Date Published:
- Journal Name:
- Frontiers in Astronomy and Space Sciences
- Volume:
- 9
- ISSN:
- 2296-987X
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract High‐latitude ionospheric convection is a useful diagnostic of solar wind‐magnetosphere interactions and nightside activity in the magnetotail. For decades, the high‐latitude convection pattern has been mapped using the Super Dual Auroral Radar Network (SuperDARN), a distribution of ground‐based radars which are capable of measuring line‐of‐sight (l‐o‐s) ionospheric flows. From the l‐o‐s measurements an estimate of the global convection can be obtained. As the SuperDARN coverage is not truly global, it is necessary to constrain the maps when the map fitting is performed. The lower latitude boundary of the convection, known as the Heppner‐Maynard boundary (HMB), provides one such constraint. In the standard SuperDARN fitting, the HMB location is determined directly from the data, but data gaps can make this challenging. In this study we evaluate if the HMB placement can be improved using data from the Active Magnetosphere and Planetary Electrodynamics Response Experiment (AMPERE), in particular for active time periods when the HMB moves to latitudes below . We find that the boundary as defined by SuperDARN and AMPERE are not always co‐located. SuperDARN performs better when the AMPERE currents are very weak (e.g., during non‐active times) and AMPERE can provide a boundary when there is no SuperDARN scatter. Using three geomagnetic storm events, we show that there is agreement between the SuperDARN and AMPERE boundaries but the SuperDARN‐derived convection boundary mostly lies equatorward of the AMPERE‐derived boundary. We find that disagreements primarily arise due to geometrical factors and a time lag in expansions and contractions of the patterns.more » « less
-
Abstract Despite widespread use of radio-echo sounding (RES) in glaciology and broad distribution of processed radar products, the glaciological community has no standard software for processing impulse RES data. Dependable, fast and collection-system/platform-independent processing flows could facilitate comparison between datasets and allow full utilization of large impulse RES data archives and new data. Here, we present ImpDAR, an open-source, cross-platform, impulse radar processor and interpreter, written primarily in Python. The utility of this software lies in its collection of established tools into a single, open-source framework. ImpDAR aims to provide a versatile standard that is accessible to radar-processing novices and useful to specialists. It can read data from common commercial ground-penetrating radars (GPRs) and some custom-built RES systems. It performs all the standard processing steps, including bandpass and horizontal filtering, time correction for antenna spacing, geolocation and migration. After processing data, ImpDAR's interpreter includes several plotting functions, digitization of reflecting horizons, calculation of reflector strength and export of interpreted layers. We demonstrate these capabilities on two datasets: deep (~3000 m depth) data collected with a custom (3 MHz) system in northeast Greenland and shallow (<100 m depth, 500 MHz) data collected with a commercial GPR on South Cascade Glacier in Washington.more » « less
-
Abstract Ultra low frequency (ULF; 1 mHz ‐ several Hz) waves are key to energy transport within the geospace system, yet their contribution to Joule heating in the upper atmosphere remains poorly quantified. This study statistically examines Joule heating associated with ionospheric ULF perturbations using Super Dual Auroral Radar Network (SuperDARN) data spanning middle to polar latitudes. Our analysis utilizes high‐time‐resolution measurements from SuperDARN high‐frequency coherent scatter radars operating in a special mode, sampling three “camping beams” approximately every 18 s. We focus on ULF perturbations within the Pc5 frequency range (1.6–6.7 mHz), estimating Joule heating rates from ionospheric electric fields derived from SuperDARN data and height‐integrated Pedersen conductance from empirical models. The analysis includes statistical characterization of Pc5 wave occurrence, electric fields, Joule heating rates, and azimuthal wave numbers. Our results reveal enhanced electric fields and Joule heating rates in the morning and pre‐midnight sectors, even though Pc5 wave occurrences peak in the afternoon. Joule heating is more pronounced in the high‐latitude morning sector during northward interplanetary magnetic field conditions, attributed to local time asymmetry in Pedersen conductance and Pc5 waves driven by Kelvin‐Helmholtz instability. Pc5 waves observed by multiple camping beams predominantly propagate westward at low azimuthal wave numbers , while high‐m waves propagate mainly eastward. Although Joule heating estimates may be underestimated due to assumptions about empirical conductance models and the underestimation of electric fields resulting from SuperDARN line‐of‐sight velocity measurements, these findings offer valuable insights into ULF wave‐related energy dissipation in the geospace system.more » « less
-
Abstract The ground‐based, high‐frequency radars of the Super Dual Auroral Radar Network (SuperDARN) observe backscatter from ionospheric field‐aligned plasma irregularities and features on the Earth's surface out to ranges of several thousand kilometers via over‐the‐horizon propagation of transmitted radio waves. Interferometric techniques can be applied to the received signals at the primary and secondary antenna arrays to measure the vertical angle of arrival, or elevation angle, for more accurate geolocation of SuperDARN observations. However, the calibration of SuperDARN interferometer measurements remains challenging for several reasons, including a 2πphase ambiguity when solving for the time delay correction factor needed to account for differences in the electrical path lengths between signals received at the two antenna arrays. We present a new technique using multi‐frequency ionospheric and ground backscatter observations for the calibration of SuperDARN interferometer data, and demonstrate its application to both historical and recent data.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Journal Article:
- https://doi.org/10.3389/fspas.2022.1022690
