skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Optimizing Sparse RFI Prediction using Deep Learning
Abstract Radio Frequency Interference (RFI) is an ever-present limiting factor among radio telescopes even in the most remote observing locations. When looking to retain the maximum amount of sensitivity and reduce contamination for Epoch of Reionization studies, the identification and removal of RFI is especially important. In addition to improved RFI identification, we must also take into account computational efficiency of the RFI-Identification algorithm as radio interferometer arrays such as the Hydrogen Epoch of Reionization Array grow larger in number of receivers. To address this, we present a Deep Fully Convolutional Neural Network (DFCN) that is comprehensive in its use of interferometric data, where both amplitude and phase information are used jointly for identifying RFI. We train the network using simulated HERA visibilities containing mock RFI, yielding a known “ground truth” dataset for evaluating the accuracy of various RFI algorithms. Evaluation of the DFCN model is performed on observations from the 67 dish build-out, HERA-67, and achieves a data throughput of 1.6× 105 HERA time-ordered 1024 channeled visibilities per hour per GPU. We determine that relative to an amplitude only network including visibility phase adds important adjacent time-frequency context which increases discrimination between RFI and Non-RFI. The inclusion of phase when predicting achieves a Recall of 0.81, Precision of 0.58, and F2 score of 0.75 as applied to our HERA-67 observations.  more » « less
Award ID(s):
1636646
PAR ID:
10110388
Author(s) / Creator(s):
; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; more » ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; « less
Date Published:
Journal Name:
Monthly Notices of the Royal Astronomical Society
ISSN:
0035-8711
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; ; ; ; ; et al (, Monthly Notices of the Royal Astronomical Society)
    ABSTRACT To mitigate the effects of Radio Frequency Interference (RFI) on the data analysis pipelines of 21 cm interferometric instruments, numerous inpaint techniques have been developed. In this paper, we examine the qualitative and quantitative errors introduced into the visibilities and power spectrum due to inpainting. We perform our analysis on simulated data as well as real data from the Hydrogen Epoch of Reionization Array (HERA) Phase 1 upper limits. We also introduce a convolutional neural network that is capable of inpainting RFI corrupted data. We train our network on simulated data and show that our network is capable of inpainting real data without requiring to be retrained. We find that techniques that incorporate high wavenumbers in delay space in their modelling are best suited for inpainting over narrowband RFI. We show that with our fiducial parameters discrete prolate spheroidal sequences (dpss) and clean provide the best performance for intermittent RFI while Gaussian progress regression (gpr) and least squares spectral analysis (lssa) provide the best performance for larger RFI gaps. However, we caution that these qualitative conclusions are sensitive to the chosen hyperparameters of each inpainting technique. We show that all inpainting techniques reliably reproduce foreground dominated modes in the power spectrum. Since the inpainting techniques should not be capable of reproducing noise realizations, we find that the largest errors occur in the noise dominated delay modes. We show that as the noise level of the data comes down, clean and dpss are most capable of reproducing the fine frequency structure in the visibilities. 
    more » « less
  2. ; (, Publications of the Astronomical Society of Australia)
    Abstract Radio-frequency interference (RFI) presents a significant obstacle to current radio interferometry experiments aimed at the Epoch of Reionization. RFI contamination is often several orders of magnitude brighter than the astrophysical signals of interest, necessitating highly precise identification and flagging. Although existing RFI flagging tools have achieved some success, the pervasive nature of this contamination leads to the rejection of excessive data volumes. In this work, we present a way to estimate an RFI emitter’s altitude using near-field corrections. Being able to obtain the precise location of such an emitter could shift the strategy from merely flagging to subtracting or peeling the RFI, allowing us to preserve a higher fraction of usable data. We conduct a preliminary study using a two-minute observation from the Murchison-Widefield Array (MWA) in which an unknown object briefly crosses the field of view, reflecting RFI signals into the array. By applying near-field corrections that bring the object into focus, we are able to estimate its approximate altitude and speed to be$$11.7$$km and 792 km/h, respectively. This allows us to confidently conclude that the object in question is in fact an airplane. We further validate our technique through the analysis of two additional RFI-containing MWA observations, where we are consistently able to identify airplanes as the source of the interference. 
    more » « less
  3. ; ; ; ; (, Monthly Notices of the Royal Astronomical Society)
    null (Ed.)
    ABSTRACT We quantify the effect of radio frequency interference (RFI) on measurements of the 21-cm power spectrum during the Epoch of Reionization (EoR). Specifically, we investigate how the frequency structure of RFI source emission generates contamination in higher order wave modes, which is much more problematic than smooth-spectrum foreground sources. Using a relatively optimistic EoR model, we find that even a single relatively dim RFI source can overwhelm the EoR power spectrum signal of $$\sim 10\, {\rm mK}^2$$ for modes $$0.1 \ \lt k \lt 2 \, h\, {\rm Mpc}^{-1}$$. If the total apparent RFI flux density in the final power spectrum integration is kept below 1 mJy, an EoR signal resembling this optimistic model should be detectable for modes $$k \lt 0.9\, h\, {\rm Mpc}^{-1}$$, given no other systematic contaminants and an error tolerance as high as 10 per cent. More pessimistic models will be more restrictive. These results emphasize the need for highly effective RFI mitigation strategies for telescopes used to search for the EoR. 
    more » « less
  4. ; ; ; ; ; ; ; ; ; ; et al (, Monthly Notices of the Royal Astronomical Society)
    ABSTRACT Combining the visibilities measured by an interferometer to form a cosmological power spectrum is a complicated process. In a delay-based analysis, the mapping between instrumental and cosmological space is not a one-to-one relation. Instead, neighbouring modes contribute to the power measured at one point, with their respective contributions encoded in the window functions. To better understand the power measured by an interferometer, we assess the impact of instrument characteristics and analysis choices on these window functions. Focusing on the Hydrogen Epoch of Reionization Array (HERA) as a case study, we find that long-baseline observations correspond to enhanced low-k tails of the window functions, which facilitate foreground leakage, whilst an informed choice of bandwidth and frequency taper can reduce said tails. With simple test cases and realistic simulations, we show that, apart from tracing mode mixing, the window functions help accurately reconstruct the power spectrum estimator of simulated visibilities. The window functions depend strongly on the beam chromaticity and less on its spatial structure – a Gaussian approximation, ignoring side lobes, is sufficient. Finally, we investigate the potential of asymmetric window functions, down-weighting the contribution of low-k power to avoid foreground leakage. The window functions presented here correspond to the latest HERA upper limits for the full Phase I data. They allow an accurate reconstruction of the power spectrum measured by the instrument and will be used in future analyses to confront theoretical models and data directly in cylindrical space. 
    more » « less
  5. ; ; ; ; ; ; ; ; ; (, The Astrophysical Journal)
    Abstract We present deep upper limits from the 2014 Murchison Widefield Array Phase I observing season, with a particular emphasis on identifying the spectral fingerprints of extremely faint radio frequency interference (RFI) contamination in the 21 cm power spectra (PS). After meticulous RFI excision involving a combination of theSSINSRFI flagger and a series of PS-based jackknife tests, our lowest upper limit on the Epoch of Reionization (EoR) 21 cm PS signal is Δ2≤ 1.61 × 104mK2atk= 0.258h Mpc−1at a redshift of 7.1 using 14.7 hr of data. By leveraging our understanding of how even fainter RFI is likely to contaminate the EoR PS, we are able to identify ultrafaint RFI signals in the cylindrical PS. Surprisingly this signature is most obvious in PS formed with less than 1 hr of data, but is potentially subdominant to other systematics in multiple-hour integrations. Since the total RFI budget in a PS detection is quite strict, this nontrivial integration behavior suggests a need to more realistically model coherently integrated ultrafaint RFI in PS measurements so that its potential contribution to a future detection can be diagnosed. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email