More Like this

1. Learning from Label Proportions by Learning with Label Noise

   Zhang, Jianxin; Wang, Yutong; Scott, Clayton (January 2022, Conference on Neural Information Processing Systems (NeurIPS 2022))

   Learning from label proportions (LLP) is a weakly supervised classification problem where data points are grouped into bags, and the label proportions within each bag are observed instead of the instance-level labels. The task is to learn a classifier to predict the labels of future individual instances. Prior work on LLP for multi-class data has yet to develop a theoretically grounded algorithm. In this work, we propose an approach to LLP based on a reduction to learning with label noise, using the forward correction (FC) loss of Patrini et al. [30]. We establish an excess risk bound and generalization error analysis for our approach, while also extending the theory of the FC loss which may be of independent interest. Our approach demonstrates improved empirical performance in deep learning scenarios across multiple datasets and architectures, compared to the leading methods. 

   more » « less
2. Learning from Label Proportions by Learning with Label Noise

   Zhang, Jianxin; Wang, Yutong; Scott, Clayton (January 2022, 36th Conference on Neural Information Processing Systems (NeurIPS 2022))

   Learning from label proportions (LLP) is a weakly supervised classification problem where data points are grouped into bags, and the label proportions within each bag are observed instead of the instance-level labels. The task is to learn a classifier to predict the labels of future individual instances. Prior work on LLP for multi-class data has yet to develop a theoretically grounded algorithm. In this work, we propose an approach to LLP based on a reduction to learning with label noise, using the forward correction (FC) loss of Patrini et al. [30]. We establish an excess risk bound and generalization error analysis for our approach, while also extending the theory of the FC loss which may be of independent interest. Our approach demonstrates improved empirical performance in deep learning scenarios across multiple datasets and architectures, compared to the leading methods. 

   more » « less
3. Deep Learning From Crowdsourced Labels: Coupled Cross-Entropy Minimization, Identifiability, and Regularization

   Ibrahim, Shahana; Nguyen, Tri; Fu, Xiao (February 2023, international conference on representation learning)

   Using noisy crowdsourced labels from multiple annotators, a deep learning-based end-to-end (E2E) system aims to learn the label correction mechanism and the neural classifier simultaneously. To this end, many E2E systems concatenate the neural classifier with multiple annotator-specific label confusion layers and co-train the two parts in a parameter-coupled manner. The formulated coupled cross-entropy minimization (CCEM)-type criteria are intuitive and work well in practice. Nonetheless, theoretical understanding of the CCEM criterion has been limited. The contribution of this work is twofold: First, performance guarantees of the CCEM criterion are presented. Our analysis reveals for the first time that the CCEM can indeed correctly identify the annotators' confusion characteristics and the desired ``ground-truth'' neural classifier under realistic conditions, e.g., when only incomplete annotator labeling and finite samples are available. Second, based on the insights learned from our analysis, two regularized variants of the CCEM are proposed. The regularization terms provably enhance the identifiability of the target model parameters in various more challenging cases. A series of synthetic and real data experiments are presented to showcase the effectiveness of our approach. 

   more » « less
4. Capturing Temporal Components for Time Series Classification

   Vavilthota, Venkata Ragavendra; Ramanathan, Ranjith; Aakur, Sathyanarayanan N (September 2024, Springer)

   Analyzing sequential data is crucial in many domains, particularly due to the abundance of data collected from the Internet of Things paradigm. Time series classification, the task of categorizing sequential data, has gained prominence, with machine learning approaches demonstrating remarkable performance on public benchmark datasets. However, progress has primarily been in designing architectures for learning representations from raw data at fixed (or ideal) time scales, which can fail to generalize to longer sequences. This work introduces a \textit{compositional representation learning} approach trained on statistically coherent components extracted from sequential data. Based on a multi-scale change space, an unsupervised approach is proposed to segment the sequential data into chunks with similar statistical properties. A sequence-based encoder model is trained in a multi-task setting to learn compositional representations from these temporal components for time series classification. We demonstrate its effectiveness through extensive experiments on publicly available time series classification benchmarks. Evaluating the coherence of segmented components shows its competitive performance on the unsupervised segmentation task. 

   more » « less
5. State of Health Estimation of Electric Vehicle Batteries Using Transformer-Based Neural Network

   https://doi.org/10.1115/1.4065762  

   Zhao, Yixin; Behdad, Sara (October 2024, Journal of Energy Resources Technology)

   Abstract Electric vehicles (EVs) are considered an environmentally friendly option compared to conventional vehicles. As the most critical module in EVs, batteries are complex electrochemical components with nonlinear behavior. On-board battery system performance is also affected by complicated operating environments. Real-time EV battery in-service status prediction is tricky but vital to enable fault diagnosis and prevent dangerous occurrences. Data-driven models with advantages in time-series analysis can be used to capture the degradation pattern from data about certain performance indicators and predict the battery states. The transformer model can capture long-range dependencies efficiently using a multi-head attention block mechanism. This paper presents the implementation of a standard transformer and an encoder-only transformer neural network to predict EV battery state of health (SOH). Based on the analysis of the lithium-ion battery from the NASA Prognostics Center of Excellence website's publicly accessible dataset, 28 features related to the charge and discharge measurement data are extracted. The features are screened using Pearson correlation coefficients. The results show that the filtered features can improve the model's accuracy and computational efficiency. The proposed standard transformer shows good performance in the SOH prediction. 

   more » « less