An official website of the United States government
We explore the statistical radio frequency interference (RFI) mitigation technique spectral kurtosis (SK) in the context of simulated realistic RFI signals. SK is a per-channel RFI detection metric that estimates the kurtosis of a collection of M power values in a single channel to discern between human-made RFI and incoherent astronomical signals of interest. We briefly test the ability of SK to flag signals with various representative modulation types, data rates, and duty cycles, as well as accumulation lengths M and multi-scale SK bin shapes. Multi-scale SK uses a rolling window to combine information from adjacent time-frequency pixels to mitigate weaknesses in single-scale SK. High data rate RFI signals with significant sidelobe emission are harder to flag, as well as signals with a 50% effective duty cycle. Multi-scale SK using at least one extra channel can detect both the center channel and side-band interference, flagging most of the signal at the expense of larger false positive rates.
more »
« less
Interference detection in radio astronomy: applying Shapiro–Wilks normality test, spectral entropy, and spectral relative entropy
Abstract Radio-frequency interference (RFI) is becoming an increasingly significant problem for most radio telescopes. Working with Green Bank Telescope data from PSR J1730+0747 in the form of complex-valued channelized voltages and their respective high-resolution power spectral densities, we evaluate a variety of statistical measures to characterize RFI. As a baseline for performance comparison, we use median absolute deviation (MAD) in complex channelized voltage data and spectral kurtosis (SK) in power spectral density data to characterize and filter out RFI. From a new perspective, we implement the Shapiro–Wilks (SW) test for normality and two information theoretical measures, spectral entropy (SE) and spectral relative entropy (SRE), and apply them to mitigate RFI. The baseline RFI mitigation algorithms are compared against our novel RFI detection algorithms to determine how effective and robust the performance is. Except for MAD, we find significant improvements in signal-to-noise ratio through the application of SE, symmetrical SRE, asymmetrical SRE, SK, and SW. These algorithms also do a good job of characterizing broad-band RFI. Time- and frequency-variable RFI signals are best detected by SK and SW tests.
more »
« less
- Award ID(s):
- 2307581
- PAR ID:
- 10565383
- Publisher / Repository:
- Oxford University Press
- Date Published:
- Journal Name:
- RAS Techniques and Instruments
- Volume:
- 3
- Issue:
- 1
- ISSN:
- 2752-8200
- Page Range / eLocation ID:
- 535 to 547
- Subject(s) / Keyword(s):
- Machine learning normality tests spectral relative entropy PSR J1713 + 0747
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract We investigate the effectiveness of the statistical radio frequency interference (RFI) mitigation technique spectral kurtosis ( SK ^ ) in the face of simulated realistic RFI signals. SK ^ estimates the kurtosis of a collection of M power values in a single channel and provides a detection metric that is able to discern between human-made RFI and incoherent astronomical signals of interest. We test the ability of SK ^ to flag signals with various representative modulation types, data rates, duty cycles, and carrier frequencies. We flag with various accumulation lengths M and implement multiscale SK ^ , which combines information from adjacent time-frequency bins to mitigate weaknesses in single-scale SK ^ . We find that signals with significant sidelobe emission from high data rates are harder to flag, as well as signals with a 50% effective duty cycle and weak signal-to-noise ratios. Multiscale SK ^ with at least one extra channel can detect both the center channel and sideband interference, flagging greater than 90% as long as the bin channel width is wider in frequency than the RFI.more » « less
-
ABSTRACT To mitigate the effects of Radio Frequency Interference (RFI) on the data analysis pipelines of 21 cm interferometric instruments, numerous inpaint techniques have been developed. In this paper, we examine the qualitative and quantitative errors introduced into the visibilities and power spectrum due to inpainting. We perform our analysis on simulated data as well as real data from the Hydrogen Epoch of Reionization Array (HERA) Phase 1 upper limits. We also introduce a convolutional neural network that is capable of inpainting RFI corrupted data. We train our network on simulated data and show that our network is capable of inpainting real data without requiring to be retrained. We find that techniques that incorporate high wavenumbers in delay space in their modelling are best suited for inpainting over narrowband RFI. We show that with our fiducial parameters discrete prolate spheroidal sequences (dpss) and clean provide the best performance for intermittent RFI while Gaussian progress regression (gpr) and least squares spectral analysis (lssa) provide the best performance for larger RFI gaps. However, we caution that these qualitative conclusions are sensitive to the chosen hyperparameters of each inpainting technique. We show that all inpainting techniques reliably reproduce foreground dominated modes in the power spectrum. Since the inpainting techniques should not be capable of reproducing noise realizations, we find that the largest errors occur in the noise dominated delay modes. We show that as the noise level of the data comes down, clean and dpss are most capable of reproducing the fine frequency structure in the visibilities.more » « less
-
Abstract The Breakthrough Listen search for intelligent life is, to date, the most extensive technosignature search of nearby celestial objects. We present a radio technosignature search of the centers of 97 nearby galaxies, observed by Breakthrough Listen at the Robert C. Byrd Green Bank Telescope. We performed a narrowband Doppler drift search using theturboSETIpipeline with a minimum signal-to-noise parameter threshold of 10, across a drift rate range of ±4 Hz s−1, with a spectral resolution of 3 Hz and a time resolution of ∼18.25 s. We removed radio frequency interference (RFI) by using an on-source/off-source cadence pattern of six observations and discarding signals with Doppler drift rates of 0. We assess factors affecting the sensitivity of the Breakthrough Listen data reduction and search pipeline using signal injection and recovery techniques and apply new methods for the investigation of the RFI environment. We present results in four frequency bands covering 1–11 GHz, and place constraints on the presence of transmitters with equivalent isotropic radiated power on the order of 1026W, corresponding to the theoretical power consumption of Kardashev Type II civilizations.more » « less
-
Emerging communication services and satellite system deployments pose heightened interference challenges for crucial passive radiometer sensors used in environmental and atmospheric sensing. Therefore, there is an urgent necessity to develop effective approaches for detecting, mitigating, and characterizing the influence of anthropogenic sources, commonly referred to as radio frequency interference (RFI) on passive Earth-observing microwave radiometers. Experimenting the co-existence of active communication and passive sensing systems would greatly benefit from a thorough and realistic dataset covering a wide range of scenarios. The insufficient availability of extensive datasets in the radio frequency (RF) domain, particularly in the context of active/passive coexistence, poses a significant obstacle to progress. This limitation is particularly notable in the context of comprehending the effectiveness of conventional model-based RFI detection approaches when applied to advanced 5th-generation (5G) wireless communication signals. This study first shows the development of an experimental passive radiometer and 5G testbed system and aims to assess the efficacy of the widely employed spectral kurtosis RFI detection approach within controlled anechoic chamber experiments. Our experimental setup comprises a fully calibrated SDR-based L-band radiometer subjected to diverse 5 G wireless signals, varying in power levels, frequency resource block group allocation, and modulation techniques. Significantly, our testbed facilitates the concurrent recording of ground truth temperatures while subjecting the radiometer to 5 G signal transmission which helps to understand the overall effect in the radiometer. This distinctive configuration provides insights into the effectiveness of traditional RFI detection models, offering valuable perspectives on the associated challenges in RFI detection.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Journal Article:
- https://doi.org/10.1093/rasti/rzae035
