More Like this

1. A demonstration of AutoOD: a self-tuning anomaly detection system

   https://doi.org/10.14778/3554821.3554880  

   Hofmann, Dennis; VanNostrand, Peter; Zhang, Huayi; Yan, Yizhou; Cao, Lei; Madden, Samuel; Rundensteiner, Elke (August 2022, Proceedings of the VLDB Endowment)

   Anomaly detection is a critical task in applications like preventing financial fraud, system malfunctions, and cybersecurity attacks. While previous research has offered a plethora of anomaly detection algorithms, effective anomaly detection remains challenging for users due to the tedious manual tuning process. Currently, model developers must determine which of these numerous algorithms is best suited for their particular domain and then must tune many parameters by hand to make the chosen algorithm perform well. This demonstration showcases AutoOD, the first unsupervised self-tuning anomaly detection system which frees users from this tedious manual tuning process. AutoOD outperforms the best un-supervised anomaly detection methods it deploys, with its performance similar to those of supervised anomaly classification models, yet without requiring ground truth labels. Our easy-to-use visual interface allows users to gain insights into AutoOD's self-tuning process and explore the underlying patterns within their datasets. 

   more » « less
2. Impact of Class Imbalance on Unsupervised Label Generation for Medicare Fraud Detection

   https://doi.org/10.1109/IRI62200.2024.00053  

   Robert_Kennedy, KL; Khoshgoftaar, Taghi M (August 2024, IEEE)

   Rubin, Stuart; Chen, Shu-Ching (Ed.)

   In this work, we use an unsupervised method for generating binary class labels in a novel context to create class labels for Medicare fraud detection. We examine how class imbalance influences the quality of these new labels and how it affects supervised classification. We use four different Medicare Part D fraud detection datasets, with the largest containing over 5 million instances. The other three datasets are sampled from the original dataset. Using Random Under-Sampling (RUS), we subsample from the majority class of the original data to produce three datasets with varying levels of class imbalance. To evaluate the performance of the newly created labels, we train a supervised classifier and evaluate its classification performance and compare it to an unsupervised anomaly detection method as a baseline. Our empirical findings indicate that the generated class labels are of high enough quality and enable effective supervised classifier training for fraud detection. Additionally, supervised classification with the new labels consistently outperforms the baseline used for comparison across all test scenarios. Further more, we observe an inverse relationship between class imbalance in the dataset and classifier performance, with AUPRC scores improving as the training dataset becomes more balanced. This work not only validates the efficacy of the synthesized class labels in labeling Medicare fraud but also shows its robustness across different degrees of class imbalance. 

   more » « less
3. Conformalized Semi-supervised Random Forest for Classification and Abnormality Detection

   Yujin, Han; Mingwenchan, Xu; Leying, Guan (May 2024, Proceedings of The 27th International Conference on Artificial Intelligence and Statistics)

   The Random Forests classifier, a widely utilized off-the-shelf classification tool, assumes training and test samples come from the same distribution as other standard classifiers. However, in safety-critical scenarios like medical diagnosis and network attack detection, discrepancies between the training and test sets, including the potential presence of novel outlier samples not appearing during training, can pose significant challenges. To address this problem, we introduce the Conformalized Semi-Supervised Random Forest (CSForest), which couples the conformalization technique Jackknife+aB with semi-supervised tree ensembles to construct a set-valued prediction 𝐶(𝑥). Instead of optimizing over the training distribution, CSForest employs unlabeled test samples to enhance accuracy and flag unseen outliers by generating an empty set. Theoretically, we establish CSForest to cover true labels for previously observed inlier classes under arbitrarily label-shift in the test data. We compare CSForest with state-of-the-art methods using synthetic examples and various real-world datasets, under different types of distribution changes in the test domain. Our results highlight CSForest’s effective prediction of inliers and its ability to detect outlier samples unique to the test data. In addition, CSForest shows persistently good performance as the sizes of the training and test sets vary. Codes of CSForest are available at https://github.com/yujinhan98/CSForest. 

   more » « less
4. A Comparative Analysis of Supervised and Unsupervised Models for Detecting GPS Spoofing Attack on UAVs

   https://doi.org/10.1109/eIT53891.2022.9813826  

   Khoei, T. Talaei; Gasimova, A.; Ahajjam, M. A.; Shamaileh, K. Al; Devabhaktuni, V.; Kaabouch, N. (May 2022, A Comparative Analysis of Supervised and Unsupervised Models for Detecting GPS Spoofing Attack on UAVs)

   With the increasing use of Unmanned Aerial Vehicles in military and civilian applications, the security of this technology has become one of the critical concerns. UAVs’ positioning and navigation activities are highly dependent on Global Positioning Systems as they provide accurate locations for these vehicles. However, due to the civilian GPS signals being open and unencrypted, malicious users can target them in multiple ways, including by launching Global Positioning System spoofing attacks. To address this security issue, numerous techniques have been proposed to detect and classify these attacks, including supervised machine learning techniques. However, no studies have focused on unsupervised models to detect these attacks. In this paper, we compare the performance of several supervised models with that of unsupervised models in terms of accuracy, probability of detection, probability of misdetection, probability of false alarm, processing time, training time, prediction time, and memory size. The supervised models are Gaussian Naïve Bayes, Classification and Regression Decision Tree, Logistic Regression, Random Forest, Linear-Support Vector Machine, and Artificial Neural Network. The unsupervised models are Principal Component Analysis, K-means clustering, and Autoencoder. The results show that the Classification and Regression Decision Tree model outperforms the other supervised and unsupervised models in detecting and classifying GPS spoofing attacks. 

   more » « less
5. Detecting Outliers in Data with Correlated Measures

   https://doi.org/10.1145/3269206.3271798  

   Kuo, Yu-Hsuan; Li, Zhenhui; Kifer, Daniel (October 2018, Proceedings of the 27th ACM International Conference on Information and Knowledge Management)

   Advances in sensor technology have enabled the collection of largescale datasets. Such datasets can be extremely noisy and often contain a significant amount of outliers that result from sensor malfunction or human operation faults. In order to utilize such data for real-world applications, it is critical to detect outliers so that models built from these datasets will not be skewed by outliers. In this paper, we propose a new outlier detection method that utilizes the correlations in the data (e.g., taxi trip distance vs. trip time). Different from existing outlier detection methods, we build a robust regression model that explicitly models the outliers and detects outliers simultaneously with the model fitting. We validate our approach on real-world datasets against methods specifically designed for each dataset as well as the state of the art outlier detectors. Our outlier detection method achieves better performances, demonstrating the robustness and generality of our method. Last, we report interesting case studies on some outliers that result from atypical events. 

   more » « less