Abstract Radio Frequency Interference (RFI) is an ever-present limiting factor among radio telescopes even in the most remote observing locations. When looking to retain the maximum amount of sensitivity and reduce contamination for Epoch of Reionization studies, the identification and removal of RFI is especially important. In addition to improved RFI identification, we must also take into account computational efficiency of the RFI-Identification algorithm as radio interferometer arrays such as the Hydrogen Epoch of Reionization Array grow larger in number of receivers. To address this, we present a Deep Fully Convolutional Neural Network (DFCN) that is comprehensive in its use of interferometric data, where both amplitude and phase information are used jointly for identifying RFI. We train the network using simulated HERA visibilities containing mock RFI, yielding a known “ground truth” dataset for evaluating the accuracy of various RFI algorithms. Evaluation of the DFCN model is performed on observations from the 67 dish build-out, HERA-67, and achieves a data throughput of 1.6× 105 HERA time-ordered 1024 channeled visibilities per hour per GPU. We determine that relative to an amplitude only network including visibility phase adds important adjacent time-frequency context which increases discrimination between RFI and Non-RFI. The inclusion of phase when predicting achieves a Recall of 0.81, Precision of 0.58, and F2 score of 0.75 as applied to our HERA-67 observations.
more »
« less
Automated RFI Flagging for HI Spectra
Manual flagging of RFI is extremely time-consuming and error-prone. We present a machine learning algorithm which automatically identifies radio frequency interference (RFI) in HI spectra. Our algorithm uses the features of polarization asymmetry (defined as |polA - polB|/[polA + polB] ) along with the skew and standard deviation of each channel over time to evaluate the presence of RFI. The algorithm was tested on hundreds of spectra taken by the Undergraduate ALFALFA Team (UAT) as part of the APPSS survey. It outperforms humans not only in speed, but in visually identifying RFI when it is weak or mimics properties of signals. This work has been supported by NSF grants AST-1211005 and AST-1637339.
more »
« less
- Award ID(s):
- 1637339
- NSF-PAR ID:
- 10097651
- Date Published:
- Journal Name:
- American Astronomical Society, AAS Meeting
- Volume:
- 233
- Page Range / eLocation ID:
- 245.22
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Abstract We present deep upper limits from the 2014 Murchison Widefield Array Phase I observing season, with a particular emphasis on identifying the spectral fingerprints of extremely faint radio frequency interference (RFI) contamination in the 21 cm power spectra (PS). After meticulous RFI excision involving a combination of the
SSINS RFI flagger and a series of PS-based jackknife tests, our lowest upper limit on the Epoch of Reionization (EoR) 21 cm PS signal is Δ2≤ 1.61 × 104mK2atk = 0.258h Mpc−1at a redshift of 7.1 using 14.7 hr of data. By leveraging our understanding of how even fainter RFI is likely to contaminate the EoR PS, we are able to identify ultrafaint RFI signals in the cylindrical PS. Surprisingly this signature is most obvious in PS formed with less than 1 hr of data, but is potentially subdominant to other systematics in multiple-hour integrations. Since the total RFI budget in a PS detection is quite strict, this nontrivial integration behavior suggests a need to more realistically model coherently integrated ultrafaint RFI in PS measurements so that its potential contribution to a future detection can be diagnosed. -
more » « less
Abstract Quasiperiodic radio frequency interference (RFI), such as those generated by telecommunication and active radar systems, is commonly encountered in radio astronomy observations. Such RFI‐contaminated signals contain hidden periodicities due to cyclic features involved in their formation (e.g., carrier frequencies, periodic keying of the amplitude, and baud rates). RFI signal characterization and its subsequent excision based on the well‐known cyclic spectrum analysis have been previously demonstrated; however, the high complexity of the algorithm and the computational cost of its implementation have limited its utility in radio astronomy, rendering less sophisticated solutions. To overcome this challenge, we present a novel method for RFI detection and mitigation based on efficient estimation of the cyclic spectrum by compressive statistical sensing (CSS) of sub‐Nyquist data. CSS performs second‐order statistical estimation such as cyclic spectrum using a reduced number of input samples, thereby enabling accelerated performance. To validate the feasibility of the proposed method, we conduct experiments with simulated data and assess the detection and mitigation results under different parameter settings, for example, interference‐to‐noise ratio, additional RFI sources, frequency resolution, and input data size. We demonstrate the real performance of the method by analyzing radio astronomy data (∼1.3 GHz) acquired with the L‐wide band receiver at the Arecibo Observatory, which is typically corrupted by active air surveillance radars located nearby. Our CSS‐based solution enables robust and efficient detection of the RFI frequency bands present in the L‐band data, and subsequent excision by blanking is also demonstrated.
-
ABSTRACT We present a sample of 706, z < 1.5 active galactic nuclei (AGNs) selected from optical photometric variability in three of the Dark Energy Survey (DES) deep fields (E2, C3, and X3) over an area of 4.64 deg2. We construct light curves using difference imaging aperture photometry for resolved sources and non-difference imaging PSF photometry for unresolved sources, respectively, and characterize the variability significance. Our DES light curves have a mean cadence of 7 d, a 6-yr baseline, and a single-epoch imaging depth of up to g ∼ 24.5. Using spectral energy distribution (SED) fitting, we find 26 out of total 706 variable galaxies are consistent with dwarf galaxies with a reliable stellar mass estimate ($M_{\ast }\lt 10^{9.5}\, {\rm M}_\odot$; median photometric redshift of 0.9). We were able to constrain rapid characteristic variability time-scales (∼ weeks) using the DES light curves in 15 dwarf AGN candidates (a subset of our variable AGN candidates) at a median photometric redshift of 0.4. This rapid variability is consistent with their low black hole (BH) masses. We confirm the low-mass AGN nature of one source with a high S/N optical spectrum. We publish our catalogue, optical light curves, and supplementary data, such as X-ray properties and optical spectra, when available. We measure a variable AGN fraction versus stellar mass and compare to results from a forward model. This work demonstrates the feasibility of optical variability to identify AGNs with lower BH masses in deep fields, which may be more ‘pristine’ analogues of supermassive BH seeds.more » « less
-
The number of extragalactic sources of HI detected in radio surveys is growing exponentially. It will soon no longer be feasible for human researchers to individually fit spectra. We present algorithms for automatically extracting the typical parameters of interest for the 21 cm HI line—recessional velocity, velocity width, and integrated flux—using neural networks. Features are produced by convolving spectra with templates generated with the Busy Function. We present the results of fitting hundreds of spectra with many different shapes, and at a wide range of signal to noise ratio. Additionally, we compare with prior methods of automated source extraction. This work has been supported by NSF grants AST-1211005 and AST-1637339.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
