An official website of the United States government
Active learning methods, like uncertainty sampling, combined with probabilistic prediction techniques have achieved success in various problems like image classification and text classification. For more complex multivariate prediction tasks, the relationships between labels play an important role in designing structured classifiers with better performance. However, computational time complexity limits prevalent probabilistic methods from effectively supporting active learning. Specifically, while non-probabilistic methods based on structured support vector ma-chines can be tractably applied to predicting cuts and bipartite matchings, conditional random fields are intractable for these structures. We propose an adversarial approach for active learning with structured prediction domains that is tractable for cuts and matching. We evaluate this approach algorithmically in two important structured prediction problems: multi-label classification and object tracking in videos. We demonstrate better accuracy and computational efficiency for our proposed method.
more »
« less
ARC: Adversarial Robust Cuts for Semi-Supervised and Multi-Label Classification
Many structured prediction tasks arising in computer vision and natural language processing tractably reduce to making minimum cost cuts in graphs with edge weights learned using maximum margin methods. Unfortunately, the hinge loss used to construct these methods often provides a particularly loose bound on the loss function of interest (e.g., the Hamming loss). We develop Adversarial Robust Cuts (ARC), an approach that poses the learning task as a minimax game between predictor and "label approximator" based on minimum cost graph cuts. Unlike maximum margin methods, this game-theoretic perspective always provides meaningful bounds on the Hamming loss. We conduct multi-label and semi-supervised binary prediction experiments that demonstrate the benefits of our approach.
more »
« less
- Award ID(s):
- 1652530
- PAR ID:
- 10060559
- Date Published:
- Journal Name:
- AAAI Conference on Artificial Intelligence
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In many structured prediction problems, complex relationships between variables are compactly defined using graphical structures. The most prevalent graphical prediction methods---probabilistic graphical models and large margin methods---have their own distinct strengths but also possess significant drawbacks. Conditional random fields (CRFs) are Fisher consistent, but they do not permit integration of customized loss metrics into their learning process. Large-margin models, such as structured support vector machines (SSVMs), have the flexibility to incorporate customized loss metrics, but lack Fisher consistency guarantees. We present adversarial graphical models (AGM), a distributionally robust approach for constructing a predictor that performs robustly for a class of data distributions defined using a graphical structure. Our approach enjoys both the flexibility of incorporating customized loss metrics into its design as well as the statistical guarantee of Fisher consistency. We present exact learning and prediction algorithms for AGM with time complexity similar to existing graphical models and show the practical benefits of our approach with experiments.more » « less
-
Estimating the uncertainty of a model’s prediction on a test point is a crucial part of ensuring reliability and calibration under distribution shifts. A minimum description length approach to this problem uses the predictive normalized maximum likelihood (pNML) distribution, which considers every possible label for a data point, and decreases confidence in a prediction if other labels are also consistent with the model and training data. In this work we propose IF-COMP, a scalable and efficient approximation of the pNML distribution that linearizes the model with a temperature- scaled Boltzmann influence function. IF-COMP can be used to produce well-calibrated predictions on test points as well as measure complexity in both labelled and unlabelled settings. We experimentally validate IF-COMP on uncertainty calibration, mislabel detection, and OOD detection tasks, where it consistently matches or beats strong baseline methods.more » « less
-
Solar flare prediction is a central problem in space weather forecasting. Existing solar flare prediction tools are mainly dependent on the GOES classification system, and models commonly use a proxy of maximum (peak) X-ray flux measurement over a particular prediction window to label instances. However, the background X-ray flux dramatically fluctuates over a solar cycle and often misleads both flare detection and flare prediction models during solar minimum, leading to an increase in false alarms. We aim to enhance the accuracy of flare prediction methods by introducing novel labeling regimes that integrate relative increases and cumulative measurements over prediction windows. Our results show that the data-driven labels can offer more precise prediction capabilities and complement the existing efforts.more » « less
-
Kirigami, the creative art of paper cutting, is a promising paradigm for mechanical metamaterials. However, to make kirigami-inspired structures a reality requires controlling the topology of kirigami to achieve connectivity and rigidity. We address this question by deriving the maximum number of cuts (minimum number of links) that still allow us to preserve global rigidity and connectivity of the kirigami. A deterministic hierarchical construction method yields an efficient topological way to control both the number of connected pieces and the total degrees of freedom. A statistical approach to the control of rigidity and connectivity in kirigami with random cuts complements the deterministic pathway, and shows that both the number of connected pieces and the degrees of freedom show percolation transitions as a function of the density of cuts (links). Together, this provides a general framework for the control of rigidity and connectivity in planar kirigami.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
