skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: χ2 from redundant calibration as a tool in the detection of faint radio-frequency interference
Radio-frequency interference detection and flagging is one of the most difficult and urgent problems in 21 cm Epoch of Reionisation research. In this work, we present χ2 from redundant calibration as a novel method for RFI detection and flagging, demonstrating it to be complementary to current state-of-the-art flagging algorithms. Beginning with a brief overview of redundant calibration and the meaning of the χ2 metric, we demonstrate a two-step RFI flagging algorithm which uses the values of this metric to detect faint RFI. We find that roughly 27.4% of observations have RFI from digital television channel 7 detected by at least one algorithm of the three tested: 18.0% of observations are flagged by the novel χ2 algorithm, 16.5% are flagged by SSINS, and 6.8% are flagged by AOFlagger (there is significant overlap in these percentages). Of the 27.4% of observations with detected DTV channel 7 RFI, 37.1% (10.2% of the total observations) are detected by χ2 alone, and not by either SSINS or AOFlagger, demonstrating a significant population of as-yet undetected RFI. We find that χ2 is able to detect RFI events which remain undetectable to SSINS and AOFlagger, especially in the domain of long-duration, weak RFI from digital television. We also discuss the shortcomings of this approach and discuss examples of RFI which seems undetectable using χ2 while being successfully flagged by SSINS and/or AOFlagger.  more » « less
Award ID(s):
2106510 1907777
PAR ID:
10654797
Author(s) / Creator(s):
;
Publisher / Repository:
Cambridge University Press
Date Published:
Journal Name:
Publications of the Astronomical Society of Australia
Volume:
41
ISSN:
1323-3580
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; (, The RFI2022 Workshop at ECMWF)
    We explore the statistical radio frequency interference (RFI) mitigation technique spectral kurtosis (SK) in the context of simulated realistic RFI signals. SK is a per-channel RFI detection metric that estimates the kurtosis of a collection of M power values in a single channel to discern between human-made RFI and incoherent astronomical signals of interest. We briefly test the ability of SK to flag signals with various representative modulation types, data rates, and duty cycles, as well as accumulation lengths M and multi-scale SK bin shapes. Multi-scale SK uses a rolling window to combine information from adjacent time-frequency pixels to mitigate weaknesses in single-scale SK. High data rate RFI signals with significant sidelobe emission are harder to flag, as well as signals with a 50% effective duty cycle. Multi-scale SK using at least one extra channel can detect both the center channel and side-band interference, flagging most of the signal at the expense of larger false positive rates. 
    more » « less
  2. ; ; (, Monthly Notices of the Royal Astronomical Society)
    ABSTRACT We explore how chromatic radio frequency interference (RFI) flags affect 21-cm power spectrum measurements. We particularly study flags that are coarser than the analysis resolution. We find that such RFI flags produce excess power in the EoR window in much the same way as residual RFI. We use Fast Holographic Deconvolution (fhd) simulations to explain this as a result of chromatic disruptions in the interferometric sampling function of the array. We also use these simulations in conjunction with Error Propagated Power Spectrum with InterLeaved Observed Noise to show that without modifying current flagging strategies or implementing extremely accurate and complete foreground subtraction, 21-cm EoR experiments will fail to make a significant detection. As a mitigation strategy, we find that circumventing the chromatic structure altogether by flagging the entire analysis band when RFI is detected is simple to implement and highly successful. This demands a detection strategy with a low false-positive rate in order to prevent excessive data loss. 
    more » « less
  3. ; (, American Astronomical Society, AAS Meeting)
    Manual flagging of RFI is extremely time-consuming and error-prone. We present a machine learning algorithm which automatically identifies radio frequency interference (RFI) in HI spectra. Our algorithm uses the features of polarization asymmetry (defined as |polA - polB|/[polA + polB] ) along with the skew and standard deviation of each channel over time to evaluate the presence of RFI. The algorithm was tested on hundreds of spectra taken by the Undergraduate ALFALFA Team (UAT) as part of the APPSS survey. It outperforms humans not only in speed, but in visually identifying RFI when it is weak or mimics properties of signals. This work has been supported by NSF grants AST-1211005 and AST-1637339. 
    more » « less
  4. ; ; (, The Astronomical Journal)
    Abstract We investigate the effectiveness of the statistical radio frequency interference (RFI) mitigation technique spectral kurtosis ( SK ^ ) in the face of simulated realistic RFI signals. SK ^ estimates the kurtosis of a collection ofMpower values in a single channel and provides a detection metric that is able to discern between human-made RFI and incoherent astronomical signals of interest. We test the ability of SK ^ to flag signals with various representative modulation types, data rates, duty cycles, and carrier frequencies. We flag with various accumulation lengthsMand implement multiscale SK ^ , which combines information from adjacent time-frequency bins to mitigate weaknesses in single-scale SK ^ . We find that signals with significant sidelobe emission from high data rates are harder to flag, as well as signals with a 50% effective duty cycle and weak signal-to-noise ratios. Multiscale SK ^ with at least one extra channel can detect both the center channel and sideband interference, flagging greater than 90% as long as the bin channel width is wider in frequency than the RFI. 
    more » « less
  5. ; (, Publications of the Astronomical Society of Australia)
    Abstract Radio-frequency interference (RFI) presents a significant obstacle to current radio interferometry experiments aimed at the Epoch of Reionization. RFI contamination is often several orders of magnitude brighter than the astrophysical signals of interest, necessitating highly precise identification and flagging. Although existing RFI flagging tools have achieved some success, the pervasive nature of this contamination leads to the rejection of excessive data volumes. In this work, we present a way to estimate an RFI emitter’s altitude using near-field corrections. Being able to obtain the precise location of such an emitter could shift the strategy from merely flagging to subtracting or peeling the RFI, allowing us to preserve a higher fraction of usable data. We conduct a preliminary study using a two-minute observation from the Murchison-Widefield Array (MWA) in which an unknown object briefly crosses the field of view, reflecting RFI signals into the array. By applying near-field corrections that bring the object into focus, we are able to estimate its approximate altitude and speed to be$$11.7$$km and 792 km/h, respectively. This allows us to confidently conclude that the object in question is in fact an airplane. We further validate our technique through the analysis of two additional RFI-containing MWA observations, where we are consistently able to identify airplanes as the source of the interference. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email