More Like this

1. Self-Supervised Learning for Online Anomaly Detection in High-Dimensional Data Streams

   https://doi.org/10.3390/electronics12091971  

   Mozaffari, Mahsa; Doshi, Keval; Yilmaz, Yasin (May 2023, Electronics)

   In this paper, we address the problem of detecting and learning anomalies in high-dimensional data-streams in real-time. Following a data-driven approach, we propose an online and multivariate anomaly detection method that is suitable for the timely and accurate detection of anomalies. We propose our method for both semi-supervised and supervised settings. By combining the semi-supervised and supervised algorithms, we present a self-supervised online learning algorithm in which the semi-supervised algorithm trains the supervised algorithm to improve its detection performance over time. The methods are comprehensively analyzed in terms of computational complexity, asymptotic optimality, and false alarm rate. The performances of the proposed algorithms are also evaluated using real-world cybersecurity datasets, that show a significant improvement over the state-of-the-art results. 

   more » « less
2. Realistic Synchrophasor Data Generation for Anomaly Detection and Event Classification

   Sajan, K. S.; Bariya, Mohini; Basak, Sanchita; Srivastava, Anurag; Dubey, Abhishek; Meier, Von A; Biswas, Gautam (January 2020, 8th Workshop on Modeling and Simulation of Cyber-Physical Energy Systems)

   The push to automate and digitize the electric grid has led to widespread installation of Phasor Measurement Units (PMUs) for improved real-time wide-area system monitoring and control. Nevertheless, transforming large volumes of highresolution PMU measurements into actionable insights remains challenging. A central challenge is creating flexible and scalable online anomaly detection in PMU data streams. PMU data can hold multiple types of anomalies arising in the physical system or the cyber system (measurements and communication networks). Increasing the grid situational awareness for noisy measurement data and Bad Data (BD) anomalies has become more and more significant. Number of machine learning, data analytics and physics based algorithms have been developed for anomaly detection, but need to be validated with realistic synchophasor data. Access to field data is very challenging due to confidentiality and security reasons. This paper presents a method for generating realistic synchrophasor data for the given synthetic network as well as event and bad data detection and classification algorithms. The developed algorithms include Bayesian and change-point techniques to identify anomalies, a statistical approach for event localization and multi-step clustering approach for event classification. Developed algorithms have been validated with satisfactory results for multiple examples of power system events including faults and load/generator/capacitor variations/switching for an IEEE test system. Set of synchrophasor data will be available publicly for other researchers. 

   more » « less
3. Few-shot Network Anomaly Detection via Cross-network Meta-learning

   Kaize Ding, Qinghai Zhou (April 2021, TheWebConf)

   null (Ed.)

   Network anomaly detection aims to find network elements (e.g., nodes, edges, subgraphs) with significantly different behaviors from the vast majority. It has a profound impact in a variety of applications ranging from finance, healthcare to social network analysis. Due to the unbearable labeling cost, existing methods are predominately developed in an unsupervised manner. Nonetheless, the anomalies they identify may turn out to be data noises or uninteresting data instances due to the lack of prior knowledge on the anomalies of interest. Hence, it is critical to investigate and develop few-shot learning for network anomaly detection. In real-world scenarios, few labeled anomalies are also easy to be accessed on similar networks from the same domain as the target network, while most of the existing works omit to leverage them and merely focus on a single network. Taking advantage of this potential, in this work, we tackle the problem of few-shot network anomaly detection by (1) proposing a new family of graph neural networks -- Graph Deviation Networks (GDN) that can leverage a small number of labeled anomalies for enforcing statistically significant deviations between abnormal and normal nodes on a network; (2) equipping the proposed GDN with a new cross- network meta-learning algorithm to realize few-shot network anomaly detection by transferring meta-knowledge from multiple auxiliary networks. Extensive experimental evaluations demonstrate the efficacy of the proposed approach on few-shot or even one-shot network anomaly detection. 

   more » « less
4. Online Multivariate Anomaly Detection and Localization for High-Dimensional Settings

   https://doi.org/10.3390/s22218264  

   Mozaffari, Mahsa; Doshi, Keval; Yilmaz, Yasin (November 2022, Sensors)

   This paper considers the real-time detection of abrupt and persistent anomalies in high-dimensional data streams. The goal is to detect anomalies quickly and accurately so that the appropriate countermeasures could be taken in time before the system possibly gets harmed. We propose a sequential and multivariate anomaly detection method that scales well to high-dimensional datasets. The proposed method follows a nonparametric, i.e., data-driven, and semi-supervised approach, i.e., trains only on nominal data. Thus, it is applicable to a wide range of applications and data types. Thanks to its multivariate nature, it can quickly and accurately detect challenging anomalies, such as changes in the correlation structure. Its asymptotic optimality and computational complexity are comprehensively analyzed. In conjunction with the detection method, an effective technique for localizing the anomalous data dimensions is also proposed. The practical use of proposed algorithms are demonstrated using synthetic and real data, and in variety of applications including seizure detection, DDoS attack detection, and video surveillance. 

   more » « less
5. Intelligent fuzzifier-based cluster validation for incomplete longitudinal digital trial data

   Ngo, Hieu; Fang, Hua; Wang, Honggang (December 2022, IEEEACM Conference on Connected Health Applications Systems and Engineering Technologies CHASE)

   Digital technology has huge potentials in transforming clinical trial research. One common issue in digital clinical trials for long-term behavioral treatments is incomplete longitudinal data, as subjects’ behavior changes over time. In this paper, we aim to improve the fuzzy clustering accuracy and stability of digital clinical trials by intelligently searching for the optimal fuzzifier, which is the key to identify the optimal number of overlapped clusters for incomplete longitudinal data. Our findings showed that integrating optimal fuzzifier searching with cluster validation can streamline the clustering process, thus enabling the intelligent fuzzy clustering procedure. 

   more » « less