In many real-world classification applications such as fake
news detection, the training data can be extremely imbalanced, which brings challenges to existing classifiers as the
majority classes dominate the loss functions of classifiers.
Oversampling techniques such as SMOTE are effective
approaches to tackle the class imbalance problem by
producing more synthetic minority samples. Despite their
success, the majority of existing oversampling methods only
consider local data distributions when generating minority
samples, which can result in noisy minority samples that do
not fit global data distributions or interleave with majority
classes. Hence, in this paper, we study the class imbalance
problem by simultaneously exploring local and global data
information since: (i) the local data distribution could give
detailed information for generating minority samples; and
(ii) the global data distribution could provide guidance to
avoid generating outliers or samples that interleave with
majority classes. Specifically, we propose a novel framework
GL-GAN, which leverages the SMOTE method to explore
local distribution in a learned latent space and employs
GAN to capture the global information, so that synthetic
minority samples can be generated under even extremely
imbalanced scenarios. Experimental results on diverse real
data sets demonstrate the effectiveness of our GL-GAN
framework in producing realistic and discriminative minority samples for improving the classification performance of
various classifiers on imbalanced training data. Our code is
available at https://github.com/wentao-repo/GL-GAN.
more »
« less
Improved Robustness to Open Set Inputs via Tempered Mixup
Supervised classification methods often assume that evaluation data is drawn from the same distribution as training data and that all classes are present for training. However, real-world classifiers must handle inputs that are far from the training distribution including samples from unknown classes. Open set robustness refers to the ability to properly label samples from previously unseen categories as novel and avoid high-confidence, incorrect predictions. Existing approaches have focused on either novel inference methods, unique training architectures, or supplementing the training data with additional background samples. Here, we propose a simple regularization technique easily applied to existing convolutional neural network architectures that improves open set robustness without a background dataset. Our method achieves state-of-the-art results on open set classification baselines and easily scales to large-scale open set classification problems.
more »
« less
- Award ID(s):
- 1909696
- NSF-PAR ID:
- 10274716
- Date Published:
- Journal Name:
- ECCV Workshop on Adversarial Robustness in the Real World (AROW)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
We investigate the role of representations and architectures for classifying 3D shapes in terms of their computational efficiency, generalization, and robustness to adversarial transformations. By varying the number of training examples and employing cross-modal transfer learning we study the role of initialization of existing deep architectures for 3D shape classification. Our analysis shows that multiview methods continue to offer the best generalization even without pretraining on large labeled image datasets, and even when trained on simplified inputs such as binary silhouettes. Furthermore, the performance of voxel-based 3D convolutional networks and point-based architectures can be improved via cross-modal transfer from image representations. Finally, we analyze the robustness of 3D shape classifiers to adversarial transformations and present a novel approach for generating adversarial perturbations of a 3D shape for multiview classifiers using a differentiable renderer. We find that point-based networks are more robust to point position perturbations while voxel-based and multiview networks are easily fooled with the addition of imperceptible noise to the input.more » « less
-
For real-world graph data, the node class distribution is inherently imbalanced and long-tailed, which naturally leads to a few-shot learning scenario with limited nodes labeled for newly emerging classes. Existing efforts are carefully designed to solve such a few-shot learning problem via data augmentation, learning transferable initialization, to name a few. However, most, if not all, of them are based on a strong assumption that all the test nodes must exclusively come from novel classes, which is impractical in real-world applications. In this paper, we study a broader and more realistic problem named generalized few-shot node classification, where the test samples can be from both novel classes and base classes. Compared with the standard fewshot node classification, this new problem imposes several unique challenges, including asymmetric classification and inconsistent preference. To counter those challenges, we propose a shot-aware graph neural network (STAGER) equipped with an uncertainty-based weight assigner module for adaptive propagation. To formulate this problem from the meta-learning perspective, we propose a new training paradigm named imbalanced episodic training to ensure the label distribution is consistent between the training and test scenarios. Experiment results on four real-world datasets demonstrate the efficacy of our model, with up to 14% accuracy improvement over baselines.more » « less
-
more » « less
Abstract Motivation Learning associations of traits with the microbial composition of a set of samples is a fundamental goal in microbiome studies. Recently, machine learning methods have been explored for this goal, with some promise. However, in comparison to other fields, microbiome data are high-dimensional and not abundant; leading to a high-dimensional low-sample-size under-determined system. Moreover, microbiome data are often unbalanced and biased. Given such training data, machine learning methods often fail to perform a classification task with sufficient accuracy. Lack of signal is especially problematic when classes are represented in an unbalanced way in the training data; with some classes under-represented. The presence of inter-correlations among subsets of observations further compounds these issues. As a result, machine learning methods have had only limited success in predicting many traits from microbiome. Data augmentation consists of building synthetic samples and adding them to the training data and is a technique that has proved helpful for many machine learning tasks.
Results In this paper, we propose a new data augmentation technique for classifying phenotypes based on the microbiome. Our algorithm, called TADA, uses available data and a statistical generative model to create new samples augmenting existing ones, addressing issues of low-sample-size. In generating new samples, TADA takes into account phylogenetic relationships between microbial species. On two real datasets, we show that adding these synthetic samples to the training set improves the accuracy of downstream classification, especially when the training data have an unbalanced representation of classes.
Availability and implementation TADA is available at https://github.com/tada-alg/TADA.
Supplementary information Supplementary data are available at Bioinformatics online.
-
In a closed world setting, classifiers are trained on examples from a number of classes and tested with unseen examples belonging to the same set of classes. However, in most real-world scenarios, a trained classifier is likely to come across novel examples that do not belong to any of the known classes. Such examples should ideally be categorized as belonging to an unknown class. The goal of an open set classifier is to anticipate and be ready to handle test examples of classes unseen during training. The classifier should be able to declare that a test example belongs to a class it does not know, and possi- bly, incorporate it into its knowledge as an example of a new class it has encoun- tered. There is some published research in open world image classification, but open set text classification remains mostly un- explored. In this paper, we investigate the suitability of Convolutional Neural Net- works (CNNs) for open set text classifi- cation. We find that CNNs are good fea- ture extractors and hence perform better than existing state-of-the-art open set clas- sifiers in smaller domains, although their open set classification abilities in general still need to be investigated.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
