More Like this

1. Gandalf: Learning label-label correlations in extreme multi-label classification via label features

   Kharbanda, Siddhant; Gupta, Devaansh; Schultheis, Erik; Banerjee, Atmadeep; Hsieh, Cho-Jui; Babbar, Rohit (August 2024, KDD)
2. Gandalf: Learning Label-label Correlations in Extreme Multi-label Classification via Label Features

   https://doi.org/10.1145/3637528.3672063  

   Kharbanda, Siddhant; Gupta, Devaansh; Schultheis, Erik; Banerjee, Atmadeep; Hsieh, Cho-Jui; Babbar, Rohit (August 2024, ACM)
3. Learning from Label Proportions by Learning with Label Noise

   Zhang, Jianxin; Wang, Yutong; Scott, Clayton (January 2022, Conference on Neural Information Processing Systems (NeurIPS 2022))

   Learning from label proportions (LLP) is a weakly supervised classification problem where data points are grouped into bags, and the label proportions within each bag are observed instead of the instance-level labels. The task is to learn a classifier to predict the labels of future individual instances. Prior work on LLP for multi-class data has yet to develop a theoretically grounded algorithm. In this work, we propose an approach to LLP based on a reduction to learning with label noise, using the forward correction (FC) loss of Patrini et al. [30]. We establish an excess risk bound and generalization error analysis for our approach, while also extending the theory of the FC loss which may be of independent interest. Our approach demonstrates improved empirical performance in deep learning scenarios across multiple datasets and architectures, compared to the leading methods. 

   more » « less
4. Learning from Label Proportions by Learning with Label Noise

   Zhang, Jianxin; Wang, Yutong; Scott, Clayton (January 2022, 36th Conference on Neural Information Processing Systems (NeurIPS 2022))

   Learning from label proportions (LLP) is a weakly supervised classification problem where data points are grouped into bags, and the label proportions within each bag are observed instead of the instance-level labels. The task is to learn a classifier to predict the labels of future individual instances. Prior work on LLP for multi-class data has yet to develop a theoretically grounded algorithm. In this work, we propose an approach to LLP based on a reduction to learning with label noise, using the forward correction (FC) loss of Patrini et al. [30]. We establish an excess risk bound and generalization error analysis for our approach, while also extending the theory of the FC loss which may be of independent interest. Our approach demonstrates improved empirical performance in deep learning scenarios across multiple datasets and architectures, compared to the leading methods. 

   more » « less
5. Knowledge Amalgamation for Multi-Label Classification via Label Dependency Transfer

   https://doi.org/10.1609/aaai.v37i8.26190  

   Thadajarassiri, Jidapa; Hartvigsen, Thomas; Gerych, Walter; Kong, Xiangnan; Rundensteiner, Elke (June 2023, Proceedings of the AAAI Conference on Artificial Intelligence)

   Multi-label classification (MLC), which assigns multiple labels to each instance, is crucial to domains from computer vision to text mining. Conventional methods for MLC require huge amounts of labeled data to capture complex dependencies between labels. However, such labeled datasets are expensive, or even impossible, to acquire. Worse yet, these pre-trained MLC models can only be used for the particular label set covered in the training data. Despite this severe limitation, few methods exist for expanding the set of labels predicted by pre-trained models. Instead, we acquire vast amounts of new labeled data and retrain a new model from scratch. Here, we propose combining the knowledge from multiple pre-trained models (teachers) to train a new student model that covers the union of the labels predicted by this set of teachers. This student supports a broader label set than any one of its teachers without using labeled data. We call this new problem knowledge amalgamation for multi-label classification. Our new method, Adaptive KNowledge Transfer (ANT), trains a student by learning from each teacher’s partial knowledge of label dependencies to infer the global dependencies between all labels across the teachers. We show that ANT succeeds in unifying label dependencies among teachers, outperforming five state-of-the-art methods on eight real-world datasets. 

   more » « less