Change point detection is widely used for finding transitions between states of data generation within a time series. Methods for change point detection currently assume this transition is instantaneous and therefore focus on finding a single point of data to classify as a change point. However, this assumption is flawed because many time series actually display short periods of transitions between different states of data generation. Previous work has shown Bayesian Online Change Point Detection (BOCPD) to be the most effective method for change point detection on a wide range of different time series. This paper explores adapting the change point detection algorithms to detect abrupt changes over short periods of time. We design a segment-based mechanism to examine a window of data points within a time series, rather than a single data point, to determine if the window captures abrupt change. We test our segment-based Bayesian change detection algorithm on 36 different time series and compare it to the original BOCPD algorithm. Our results show that, for some of these 36 time series, the segment-based approach for detecting abrupt changes can much more accurately identify change points based on standard metrics.
more »
« less
A Backward Procedure for Change-point Detection with Application to Copy Number Variation Detection.
Change-point detection regains much attention recently for analyzing array or sequencing data for copy number variation (CNV) detection. In such applications, the true signals are typically very short and buried in the long data sequence, which makes it challenging to identify the variations efficiently and accurately. In this article, we propose a new change-point detection method, a backward procedure, which is not only fast and simple enough to exploit high-dimensional data but also performs very well for detecting short signals. Although motivated by CNV detection, the backward procedure is generally applicable to assorted change-point problems that arise in a variety of scientific applications. It is illustrated by both simulated and real CNV data that the backward detection has clear advantages over other competing methods, especially when the true signal is short.
more »
« less
- Award ID(s):
- 1722691
- PAR ID:
- 10295536
- Editor(s):
- Yao, Fang
- Date Published:
- Journal Name:
- The Canadian journal of statistics
- Volume:
- 48
- Issue:
- 3
- ISSN:
- 1708-945X
- Page Range / eLocation ID:
- 366-385
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Delta blue intensity is a commonly used method to correct for the heartwood-sapwood color change in blue intensity (BI) measurements. It is based on the assumption that the heartwood-sapwood color change is similar in both earlywood and latewood. This assumption has not been supported physiologically. Furthermore, delta BI may confound the climate signals in earlywood and latewood BI as it is technically a linear combination of the other two. Here, instead of using delta BI, we used change point detection to identify the heartwood-sapwood transition, and corrected for the color change by rescaling the mean and variance of BI measurements after the transition to those immediately before. We tested three different change point detection methods and found that they agreed well with one another. Importantly, our approach preserves the climate signals in both earlywood and latewood BI data, while delta BI causes a total loss of climate signals in our test case. Therefore, we suggest that change point detection should be used instead of delta BI to account for the heartwood-sapwood color change.more » « less
-
Signal anomaly detection is commonly used to detect rogue or unexpected signals. It has many applications in interference mitigation, wireless security, optimized spectrum allocation, and radio coordination. Our work proposes a new method for anomaly detection on signal detection metadata using generative adversarial network output processed by a long short term memory recurrent neural network. We provide a performance analysis and comparison to baseline methods, and demonstrate that through the usage of metadata for analytics, we can provide robust detection, while also minimizing computation and bandwidth, and generalizing to numerous effects which differs from many prior works that focus on A.D. based signal processing on the raw RF sample data.more » « less
-
Copy number variants (CNVs) are regions of the genome that vary in integer copy number. CNVs, which comprise both amplifications and deletions of DNA sequence, have been identified across all domains of life, from bacteria and archaea to plants and animals. CNVs are an important source of genetic diversity, and can drive rapid adaptive evolution and progression of heritable and somatic human diseases, such as cancer. However, despite their evolutionary importance and clinical relevance, CNVs remain understudied compared to single-nucleotide variants (SNVs). This is a consequence of the inherent difficulties in detecting CNVs at low-to-intermediate frequencies in heterogeneous populations of cells. Here, we discuss molecular methods used to detect CNVs, the limitations associated with using these techniques, and the application of new and emerging technologies that present solutions to these challenges. The goal of this short review and perspective is to highlight aspects of CNV biology that are understudied and define avenues for further research that address specific gaps in our knowledge of these complex alleles. We describe our recently developed method for CNV detection in which a fluorescent gene functions as a single-cell CNV reporter and present key findings from our evolution experiments in Saccharomyces cerevisiae. Using a CNV reporter, we found that CNVs are generated at a high rate and undergo selection with predictable dynamics across independently evolving replicate populations. Many CNVs appear to be generated through DNA replication-based processes that are mediated by the presence of short, interrupted, inverted-repeat sequences. Our results have important implications for the role of CNVs in evolutionary processes and the molecular mechanisms that underlie CNV formation. We discuss the possible extension of our method to other applications, including tracking the dynamics of CNVs in models of human tumors.more » « less
-
Abstract The research reported in this article is concerned with the question of detecting and subsequently determining the endpoint in a long-stretch, ultraprecision surface polishing process. While polishing endpoint detection has attracted much attention for several decades in the chemical-mechanical planarization of semiconductor wafer polishing processes, the uniqueness of the surface polishing process under our investigation calls for novel solutions. To tackle the research challenges, we develop both an offline model and an online detection method. The offline model is a functional regression that relates the vibration signals to the surface roughness, whereas the online procedure is a change-point detection method that detects the energy turning points in the vibration signals. Our study reveals a number of insights. The offline functional regression model shows clearly that the polishing process progresses in three states, including a saturation phase, over which the polishing action could be substantially shortened. The online detection method signals in real-time when to break a polishing cycle and to institute a follow-up inspection, rather than letting the machine engage in an overpolishing cycle for too long. When implemented properly, both sets of insights and the corresponding methods could lead to substantial savings in polishing time and energy and significantly improve the throughput of such polishing processes without inadvertently affecting the quality of the final polish.more » « less
An official website of the United States government

