skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Tutorial on running median subtraction filter with application to searches for exotic field transients in multi-messenger astronomy
Running Median Subtraction Filter (RMSF) is a robust statistical tool for removing slowly varying baselines in data streams containing transients (short-duration signals) of interest. In this work, we explore the RMSF performance and properties using simulated time series and analytical methods. We study the RMSF fidelity in preserving the signal of interest in the data using (i) a Gaussian pulse and (ii) a transient oscillatory signal. Such signals may be generated by hypothetical exotic low-mass fields (ELFs) associated with intense astrophysical events like binary black hole or neutron star mergers. We consider and assess RMSF as a candidate method to extract transient ELF signals. RMSF operates by sliding a window across the data and subtracting the median value within each window from the data points. With a suitable choice of running window size, RMSF effectively filters out baseline variations without compromising the integrity of transients. The RMSF window width is a critical parameter: it must be wide enough to encompass a short transient but narrow enough to remove the slowly varying baseline. We show that the RMSF removes the mean of a normally distributed white noise while preserving its variance and higher-order moments in the limit of large windows. In addition, RMSF does not color the white noise stream, that is, it does not induce any significant correlation in the filtered data. Ideally, a filter would preserve both the signal of interest and the statistical characteristics of the stochastic component of the data, while removing the background clutter and outliers. We find the RMSF to satisfy these practical criteria for data preprocessing. While we rigorously prove several RMSF properties, the paper is organized as a tutorial with multiple illustrations of RMSF applications.  more » « less
Award ID(s):
2207546
PAR ID:
10587958
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
American Institute of Physics
Date Published:
Journal Name:
AIP Advances
Volume:
15
Issue:
5
ISSN:
2158-3226
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. SUMMARY Seismograms contain multiple sources of seismic waves, from distinct transient signals such as earthquakes to continuous ambient seismic vibrations such as microseism. Ambient vibrations contaminate the earthquake signals, while the earthquake signals pollute the ambient noise’s statistical properties necessary for ambient-noise seismology analysis. Separating ambient noise from earthquake signals would thus benefit multiple seismological analyses. This work develops a multitask encoder–decoder network named WaveDecompNet to separate transient signals from ambient signals directly in the time domain for 3-component seismograms. We choose the active-volcanic Big Island in Hawai’i as a natural laboratory given its richness in transients (tectonic and volcanic earthquakes) and diffuse ambient noise (strong microseism). The approach takes a noisy 3-component seismogram as input and independently predicts the 3-component earthquake and noise waveforms. The model is trained on earthquake and noise waveforms from the STandford EArthquake Dataset (STEAD) and on the local noise of seismic station IU.POHA. We estimate the network’s performance by using the explained variance metric on both earthquake and noise waveforms. We explore different neural network designs for WaveDecompNet and find that the model with long-short-term memory (LSTM) performs best over other structures. Overall, we find that WaveDecompNet provides satisfactory performance down to a signal-to-noise ratio (SNR) of 0.1. The potential of the method is (1) to improve broad-band SNR of transient (earthquake) waveforms and (2) to improve local ambient noise to monitor the Earth’s structure using ambient noise signals. To test this, we apply a short-time average to a long-time average filter and improve the number of detected events. We also measure single-station cross-correlation functions of the recovered ambient noise and establish their improved coherence through time and over different frequency bands. We conclude that WaveDecompNet is a promising tool for a broad range of seismological research. 
    more » « less
  2. Abstract Radio-frequency interference (RFI) is becoming an increasingly significant problem for most radio telescopes. Working with Green Bank Telescope data from PSR J1730+0747 in the form of complex-valued channelized voltages and their respective high-resolution power spectral densities, we evaluate a variety of statistical measures to characterize RFI. As a baseline for performance comparison, we use median absolute deviation (MAD) in complex channelized voltage data and spectral kurtosis (SK) in power spectral density data to characterize and filter out RFI. From a new perspective, we implement the Shapiro–Wilks (SW) test for normality and two information theoretical measures, spectral entropy (SE) and spectral relative entropy (SRE), and apply them to mitigate RFI. The baseline RFI mitigation algorithms are compared against our novel RFI detection algorithms to determine how effective and robust the performance is. Except for MAD, we find significant improvements in signal-to-noise ratio through the application of SE, symmetrical SRE, asymmetrical SRE, SK, and SW. These algorithms also do a good job of characterizing broad-band RFI. Time- and frequency-variable RFI signals are best detected by SK and SW tests. 
    more » « less
  3. In this paper we investigate the impact of transient noise artifacts, or glitches, on gravitational- wave inference from ground-based interferometer data, and test how modeling and subtracting these glitches affects the inferred parameters. Due to their time-frequency morphology, broadband glitches cause moderate to significant biasing of posterior distributions away from true values. In contrast, narrowband glitches induce negligible biasing effects, due to distinct signal and glitch morphologies. We inject simulated binary black hole signals into data containing three occurring glitch types from past LIGO-Virgo observing runs, and reconstruct both signal and glitch waveforms using BayesWave, a wavelet-based Bayesian analysis. We apply the standard LIGO-Virgo-KAGRA deglitching pro- cedure to the detector data, which consists of subtracting from calibrated LIGO data the glitch waveform estimated by the joint BayesWave inference. We produce posterior distributions on the parameters of the injected signal before and after subtracting the glitch, and we show that removing the transient noise effectively mitigates bias from broadband glitches. This study provides a baseline validation of existing techniques, while demonstrating waveform reconstruction improvements to the Bayesian algorithm for robust astrophysical characterization in glitch-prone detector data. 
    more » « less
  4. Giove, Federico (Ed.)
    Resting-state blood-oxygen-level-dependent (BOLD) signal acquired through functional magnetic resonance imaging is a proxy of neural activity and a key mechanism for assessing neurological conditions. Therefore, practical tools to filter out artefacts that can compromise the assessment are required. On the one hand, a variety of tailored methods to preprocess the data to deal with identified sources of noise (e.g., head motion, heart beating, and breathing, just to mention a few) are in place. But, on the other hand, there might be unknown sources of unstructured noise present in the data. Therefore, to mitigate the effects of such unstructured noises, we propose a model-based filter that explores the statistical properties of the underlying signal (i.e., long-term memory). Specifically, we consider autoregressive fractional integrative process filters. Remarkably, we provide evidence that such processes can model the signals at different regions of interest to attain stationarity. Furthermore, we use a principled analysis where a ground-truth signal with statistical properties similar to the BOLD signal under the injection of noise is retrieved using the proposed filters. Next, we considered preprocessed (i.e., the identified sources of noise removed) resting-state BOLD data of 98 subjects from the Human Connectome Project. Our results demonstrate that the proposed filters decrease the power in the higher frequencies. However, unlike the low-pass filters, the proposed filters do not remove all high-frequency information, instead they preserve process-related higher frequency information. Additionally, we considered four different metrics (power spectrum, functional connectivity using the Pearson’s correlation, coherence, and eigenbrains) to infer the impact of such filter. We provided evidence that whereas the first three keep most of the features of interest from a neuroscience perspective unchanged, the latter exhibits some variations that could be due to the sporadic activity filtered out. 
    more » « less
  5. This article presents a standardized alternative to the traditional phase cycling approach employed by the overwhelming majority of contemporary Nuclear Magnetic Resonance (NMR) research. On well-tested, stable NMR systems running well-tested pulse sequences in highly optimized, homogeneous magnetic fields, the hardware and/or software responsible for traditional phase cycling quickly isolate a meaningful subset of data by averaging and discarding between 3/4 and 127/128 of the digitized data. In contrast, the new domain colored coherence transfer (DCCT) approach enables the use of all the information acquired from all transients. This approach proves to be particularly useful where multiple coherence pathways are required, or for improving the signal when the magnetic fields are inhomogeneous and unstable. For example, the authors’ interest in the nanoscale heterogeneities of hydration dynamics demands increasingly sophisticated and automated measurements deploying Overhauser Dynamic Nuclear Polarization (ODNP) in low-field electromagnets, where phase cycling and signal averaging perform suboptimally. This article demonstrates the capabilities of DCCT on ODNP data and with a collection of algorithms that provide robust phasing, avoidance of baseline distortion, and the ability to realize relatively weak signals amid background noise through signal-averaged correlation alignment. The DCCT schema works by combining a multidimensional organization of phase cycled data with a specific methodology for visualizing the resulting complex-valued data. It could be extended to other forms of coherent spectroscopy seeking to analyze multiple coherence transfer pathways. 
    more » « less