The problem of sequential change diagnosis is considered, where a sequence of independent random elements is accessed sequentially, there is an abrupt change in its distribution at some unknown time, and there are two main operational goals: to quickly detect the change and to accurately identify the post-change distribution among a finite set of alternatives. A standard algorithm is considered, which does not explicitly address the isolation task and raises an alarm as soon as the CuSum statistic that corresponds to one of the post-change alternatives exceeds a certain threshold. It is shown that in certain cases, such as the so-called multichannel problem, this algorithm controls the worst-case conditional probability of false isolation and minimizes Lorden’s criterion, for every possible post-change distribution, to a first-order asymptotic approximation as the false alarm rate goes to zero sufficiently faster than the worst-case conditional probability of false isolation. These theoretical results are also illustrated with a numerical study.
more »
« less
A modified CUSUM test to control postoutbreak false alarms
The cumulative sum (CUSUM) control chart is a method for detecting whether the mean of a time series process has shifted beyond some tolerance (ie, is out of control). Originally developed in an industrial process control setting, the CUSUM statistic is typically reset to zero once a process is discovered to be out of control since the industrial process is then recalibrated to be in control. The CUSUM method is also used to detect disease outbreaks in prospective disease surveillance, with a disease outbreak coinciding with an out‐of‐control process. In a disease surveillance setting, resetting the CUSUM statistic is unrealistic, and a nonrestarting CUSUM chart is used instead. In practice, the nonrestarting CUSUM provides more information but suffers from a high false alarm rate following the end of an outbreak. In this paper, we propose a modified hypothesis test for use with the nonrestarting CUSUM when testing whether a process is out of control. By simulating statistics conditional on the presence of an out‐of‐control process in recent time periods, we are able to retain the CUSUM's power to detect an out‐of‐control process while controlling the post–out‐of‐control false alarm rate at the desired level. We demonstrate this method using data on a
- PAR ID:
- 10460909
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- Statistics in Medicine
- Volume:
- 38
- Issue:
- 11
- ISSN:
- 0277-6715
- Format(s):
- Medium: X Size: p. 2047-2058
- Size(s):
- p. 2047-2058
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Abstract In practice, sequential processes often have gradual changes in their process distributions over time. This is related to the drift detection problem in statistical process control. In the literature, there have been some existing discussions on this problem. But, most existing methods are designed based on the assumption that the related drift is linear or have another specific pattern. In reality, however, such specified patterns may not be valid. In this paper, we suggest an adaptive cumulative sum (CUSUM) chart to handle the drift detection problem with a flexible drift pattern. The new method integrates the general framework to construct a CUSUM chart based on the generalized likelihood ratio statistic and estimation of a shift size by the exponentially weighted least square regression procedure. Simulation studies show that the proposed method is effective in various cases considered. The new method is also illustrated using an example about the exchange rates between Indian Rupees and US Dollars.
-
Astley, Susan M ; Chen, Weijie (Ed.)Devices enabled by artificial intelligence (AI) and machine learning (ML) are being introduced for clinical use at an accelerating pace. In a dynamic clinical environment, these devices may encounter conditions different from those they were developed for. The statistical data mismatch between training/initial testing and production is often referred to as data drift. Detecting and quantifying data drift is significant for ensuring that AI model performs as expected in clinical environments. A drift detector signals when a corrective action is needed if the performance changes. In this study, we investigate how a change in the performance of an AI model due to data drift can be detected and quantified using a cumulative sum (CUSUM) control chart. To study the properties of CUSUM, we first simulate different scenarios that change the performance of an AI model. We simulate a sudden change in the mean of the performance metric at a change-point (change day) in time. The task is to quickly detect the change while providing few false-alarms before the change-point, which may be caused by the statistical variation of the performance metric over time. Subsequently, we simulate data drift by denoising the Emory Breast Imaging Dataset (EMBED) after a pre-defined change-point. We detect the change-point by studying the pre- and post-change specificity of a mammographic CAD algorithm. Our results indicate that with the appropriate choice of parameters, CUSUM is able to quickly detect relatively small drifts with a small number of false-positive alarms.more » « less
-
The goal of this work is to detect flaw formation in wire arc additive manufacturing (WAAM). This process uses an electric arc as the energy source in order to melt metallic wire and deposit the new material, similar to metal inert gas (MIG) welding. Industry has been slow to adopt WAAM due to the lack of process consistency and reliability. The WAAM process is susceptible to a multitude of stochastic disturbances that cause instability in the electric arc. These arc instabilities eventually lead to flaw formation such as porosity, spatter, and excessive deviations in the desired geometry. Therefore, the objective of this work is to detect flaw formation using in-situ acoustic (sound) data from a microphone installed near the electric arc. This data was processed using a novel wavelet integrated graph theory approach. This approach detected the onset of multiple types of flaw formations with a false alarm rate of less than 2%. Using this method, this work demonstrates the potential for in-situ monitoring and flaw detection of the WAAM process in a computationally tractable manner.more » « less
-
In many real-world applications of monitoring multivariate spatio-temporal data that are non-stationary over time, one is often interested in detecting hot-spots with spatial sparsity and temporal consistency, instead of detecting system-wise changes as in traditional statistical process control (SPC) literature. In this paper, we propose an efficient method to detect hot-spots through tensor decomposition, and our method has three steps. First, we fit the observed data into a Smooth Sparse Decomposition Tensor (SSD-Tensor) model that serves as a dimension reduction and de-noising technique: it is an additive model decomposing the original data into: smooth but non-stationary global mean, sparse local anomalies, and random noises. Next, we estimate model parameters by the penalized framework that includes Least Absolute Shrinkage and Selection Operator (LASSO) and fused LASSO penalty. An efficient recursive optimization algorithm is developed based on Fast Iterative Shrinkage Thresholding Algorithm (FISTA). Finally, we apply a Cumulative Sum (CUSUM) Control Chart to monitor model residuals after removing global means, which helps to detect when and where hot-spots occur. To demonstrate the usefulness of our proposed SSD-Tensor method, we compare it with several other methods including scan statistics, LASSO-based, PCA-based, T2-based control chart in extensive numerical simulation studies and a real crime rate dataset.more » « less
