An official website of the United States government
Abstract Radio Frequency Interference (RFI) is an ever-present limiting factor among radio telescopes even in the most remote observing locations. When looking to retain the maximum amount of sensitivity and reduce contamination for Epoch of Reionization studies, the identification and removal of RFI is especially important. In addition to improved RFI identification, we must also take into account computational efficiency of the RFI-Identification algorithm as radio interferometer arrays such as the Hydrogen Epoch of Reionization Array grow larger in number of receivers. To address this, we present a Deep Fully Convolutional Neural Network (DFCN) that is comprehensive in its use of interferometric data, where both amplitude and phase information are used jointly for identifying RFI. We train the network using simulated HERA visibilities containing mock RFI, yielding a known “ground truth” dataset for evaluating the accuracy of various RFI algorithms. Evaluation of the DFCN model is performed on observations from the 67 dish build-out, HERA-67, and achieves a data throughput of 1.6× 105 HERA time-ordered 1024 channeled visibilities per hour per GPU. We determine that relative to an amplitude only network including visibility phase adds important adjacent time-frequency context which increases discrimination between RFI and Non-RFI. The inclusion of phase when predicting achieves a Recall of 0.81, Precision of 0.58, and F2 score of 0.75 as applied to our HERA-67 observations.
more »
« less
Characterization of inpaint residuals in interferometric measurements of the epoch of reionization
ABSTRACT To mitigate the effects of Radio Frequency Interference (RFI) on the data analysis pipelines of 21 cm interferometric instruments, numerous inpaint techniques have been developed. In this paper, we examine the qualitative and quantitative errors introduced into the visibilities and power spectrum due to inpainting. We perform our analysis on simulated data as well as real data from the Hydrogen Epoch of Reionization Array (HERA) Phase 1 upper limits. We also introduce a convolutional neural network that is capable of inpainting RFI corrupted data. We train our network on simulated data and show that our network is capable of inpainting real data without requiring to be retrained. We find that techniques that incorporate high wavenumbers in delay space in their modelling are best suited for inpainting over narrowband RFI. We show that with our fiducial parameters discrete prolate spheroidal sequences (dpss) and clean provide the best performance for intermittent RFI while Gaussian progress regression (gpr) and least squares spectral analysis (lssa) provide the best performance for larger RFI gaps. However, we caution that these qualitative conclusions are sensitive to the chosen hyperparameters of each inpainting technique. We show that all inpainting techniques reliably reproduce foreground dominated modes in the power spectrum. Since the inpainting techniques should not be capable of reproducing noise realizations, we find that the largest errors occur in the noise dominated delay modes. We show that as the noise level of the data comes down, clean and dpss are most capable of reproducing the fine frequency structure in the visibilities.
more »
« less
- Award ID(s):
- 1836019
- PAR ID:
- 10466705
- Author(s) / Creator(s):
- ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; more »
- Publisher / Repository:
- Oxford University Press
- Date Published:
- Journal Name:
- Monthly Notices of the Royal Astronomical Society
- Volume:
- 520
- Issue:
- 4
- ISSN:
- 0035-8711
- Page Range / eLocation ID:
- 5552 to 5572
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
ABSTRACT Combining the visibilities measured by an interferometer to form a cosmological power spectrum is a complicated process. In a delay-based analysis, the mapping between instrumental and cosmological space is not a one-to-one relation. Instead, neighbouring modes contribute to the power measured at one point, with their respective contributions encoded in the window functions. To better understand the power measured by an interferometer, we assess the impact of instrument characteristics and analysis choices on these window functions. Focusing on the Hydrogen Epoch of Reionization Array (HERA) as a case study, we find that long-baseline observations correspond to enhanced low-k tails of the window functions, which facilitate foreground leakage, whilst an informed choice of bandwidth and frequency taper can reduce said tails. With simple test cases and realistic simulations, we show that, apart from tracing mode mixing, the window functions help accurately reconstruct the power spectrum estimator of simulated visibilities. The window functions depend strongly on the beam chromaticity and less on its spatial structure – a Gaussian approximation, ignoring side lobes, is sufficient. Finally, we investigate the potential of asymmetric window functions, down-weighting the contribution of low-k power to avoid foreground leakage. The window functions presented here correspond to the latest HERA upper limits for the full Phase I data. They allow an accurate reconstruction of the power spectrum measured by the instrument and will be used in future analyses to confront theoretical models and data directly in cylindrical space.more » « less
-
Passive Remote Sensing services are indispensable in modern society because of the applications related to climate studies and earth science. Among those, NASA’s Soil Moisture Active and Passive (SMAP) mission provides an essential climate variable such as the moisture content of the soil by using microwave radiation within protected band over 1400-1427 MHz. However, because of the increasing active wireless technologies such as Internet of Things (IoT), unmanned aerial vehicles (UAV), and 5G wireless communication, the SMAP’s passive observations are expected to experience an increasing number of Radio Frequency Interference (RFI). RFI is a well-documented issue and SMAP has a ground processing unit dedicated to tackling this issue. However, advanced techniques are needed to tackle the increasing RFI problem for passive sensing systems and to jointly coexist communication and sensing systems. In this paper, we apply a deep learning approach where a novel Convolutional Neural Network (CNN) architecture for both RFI detection and mitigation is employed. SMAP Level 1A spectrogram of antenna counts and various moments data are used as the inputs to the deep learning architecture. We simulate different types of RFI sources such as pulsed, CW or wideband anthropogenic signals. We then use artificially corrupted SMAP Level 1B antenna measurements in conjunction with RFI labels to train the learning architecture. While the learned detection network classifies input spectrograms as RFI or no-RFI cases, the mitigation network reconstructs the RFI mitigated antenna temperature images. The proposed learning framework both takes advantage of the existing SMAP data and the simulated RFI scenarios. Future remote sensing systems such as radiometers will suffer an increasing RFI problem and spectrum sharing and techniques that will allow coexistance of sensing and communication systems will be utmost importance for both parties. RFI detection and mitigation will remain a prerequisite for these radiometers and the proposed deep learning approach has the potential to provide an additional perspective to existing solutions. We will present detailed analysis on the selected deep learning architecture, obtained RFI detection accuracy levels and RFI mitigation performance.more » « less
-
null (Ed.)Ultrasound B-Mode images are created from data obtained from each element in the transducer array in a process called beamforming. The beamforming goal is to enhance signals from specified spatial locations, while reducing signal from all other locations. On clinical systems, beamforming is accomplished with the delay-and-sum (DAS) algorithm. DAS is efficient but fails in patients with high noise levels, so various adaptive beamformers have been proposed. Recently, deep learning methods have been developed for this task. With deep learning methods, beamforming is typically framed as a regression problem, where clean, ground-truth data is known, and usually simulated. For in vivo data, however, it is extremely difficult to collect ground truth information, and deep networks trained on simulated data underperform when applied to in vivo data, due to domain shift between simulated and in vivo data. In this work, we show how to correct for domain shift by learning deep network beamformers that leverage both simulated data, and unlabeled in vivo data, via a novel domain adaption scheme. A challenge in our scenario is that domain shift exists both for noisy input, and clean output. We address this challenge by extending cycle-consistent generative adversarial networks, where we leverage maps between synthetic simulation and real in vivo domains to ensure that the learned beamformers capture the distribution of both noisy and clean in vivo data. We obtain consistent in vivo image quality improvements compared to existing beamforming techniques, when applying our approach to simulated anechoic cysts and in vivo liver data.more » « less
-
Abstract We present the first event-horizon-scale images and spatiotemporal analysis of Sgr A* taken with the Event Horizon Telescope in 2017 April at a wavelength of 1.3 mm. Imaging of Sgr A* has been conducted through surveys over a wide range of imaging assumptions using the classical CLEAN algorithm, regularized maximum likelihood methods, and a Bayesian posterior sampling method. Different prescriptions have been used to account for scattering effects by the interstellar medium toward the Galactic center. Mitigation of the rapid intraday variability that characterizes Sgr A* has been carried out through the addition of a “variability noise budget” in the observed visibilities, facilitating the reconstruction of static full-track images. Our static reconstructions of Sgr A* can be clustered into four representative morphologies that correspond to ring images with three different azimuthal brightness distributions and a small cluster that contains diverse nonring morphologies. Based on our extensive analysis of the effects of sparse ( u , v )-coverage, source variability, and interstellar scattering, as well as studies of simulated visibility data, we conclude that the Event Horizon Telescope Sgr A* data show compelling evidence for an image that is dominated by a bright ring of emission with a ring diameter of ∼50 μ as, consistent with the expected “shadow” of a 4 × 10 6 M ⊙ black hole in the Galactic center located at a distance of 8 kpc.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1093/mnras/stad441
