More Like this

1. Concept Learners for Few-Shot Learning

   Cao, Kaidi; Brbic, Maria; Leskovec, Jure (January 2021, International Conference on Learning Representations (ICLR))

   null (Ed.)

   Developing algorithms that are able to generalize to a novel task given only a few labeled examples represents a fundamental challenge in closing the gap between machine- and human-level performance. The core of human cognition lies in the structured, reusable concepts that help us to rapidly adapt to new tasks and provide reasoning behind our decisions. However, existing meta-learning methods learn complex representations across prior labeled tasks without imposing any structure on the learned representations. Here we propose COMET, a meta-learning method that improves generalization ability by learning to learn along human-interpretable concept dimensions. Instead of learning a joint unstructured metric space, COMET learns mappings of high-level concepts into semi-structured metric spaces, and effectively combines the outputs of independent concept learners. We evaluate our model on few-shot tasks from diverse domains, including fine-grained image classification, document categorization and cell type annotation on a novel dataset from a biological domain developed in our work. COMET significantly outperforms strong meta-learning baselines, achieving 6–15% relative improvement on the most challenging 1-shot learning tasks, while unlike existing methods providing interpretations behind the model’s predictions. 

   more » « less
2. Concept Learners for Few-Shot Learning

   Cao, Kaidi; Brbic, Maria; Leskovec, Jure. (January 2021, International Conference on Learning Representation (ICLR))

   null (Ed.)

   Developing algorithms that are able to generalize to a novel task given only a few labeled examples represents a fundamental challenge in closing the gap between machine- and human-level performance. The core of human cognition lies in the structured, reusable concepts that help us to rapidly adapt to new tasks and provide reasoning behind our decisions. However, existing meta-learning methods learn complex representations across prior labeled tasks without imposing any structure on the learned representations. Here we propose COMET, a meta-learning method that improves generalization ability by learning to learn along human-interpretable concept dimensions. Instead of learning a joint unstructured metric space, COMET learns mappings of high-level concepts into semi-structured metric spaces, and effectively combines the outputs of independent concept learners. We evaluate our model on few-shot tasks from diverse domains, including fine-grained image classification, document categorization and cell type annotation on a novel dataset from a biological domain developed in our work. COMET significantly outperforms strong meta-learning baselines, achieving 6–15% relative improvement on the most challenging 1-shot learning tasks, while unlike existing methods providing interpretations behind the model’s predictions. 

   more » « less
3. An Evaluative Measure of Clustering Methods Incorporating Hyperparameter Sensitivity

   https://doi.org/10.1609/aaai.v36i7.20747  

   Mishra, Siddhartha; Monath, Nicholas; Boratko, Michael; Kobren, Ariel; McCallum, Andrew (June 2022, Proceedings of the AAAI Conference on Artificial Intelligence)

   Clustering algorithms are often evaluated using metrics which compare with ground-truth cluster assignments, such as Rand index and NMI. Algorithm performance may vary widely for different hyperparameters, however, and thus model selection based on optimal performance for these metrics is discordant with how these algorithms are applied in practice, where labels are unavailable and tuning is often more art than science. It is therefore desirable to compare clustering algorithms not only on their optimally tuned performance, but also some notion of how realistic it would be to obtain this performance in practice. We propose an evaluation of clustering methods capturing this ease-of-tuning by modeling the expected best clustering score under a given computation budget. To encourage the adoption of the proposed metric alongside classic clustering evaluations, we provide an extensible benchmarking framework. We perform an extensive empirical evaluation of our proposed metric on popular clustering algorithms over a large collection of datasets from different domains, and observe that our new metric leads to several noteworthy observations. 

   more » « less
4. The Connection Between Popularity Bias, Calibration, and Fairness in Recommendation

   https://doi.org/10.1145/3383313.3418487  

   Abdollahpouri, Himan; Mansoury, Masoud; Burke, Robin; Mobasher, Bamshad (September 2020, RecSys '20: Fourteenth ACM Conference on Recommender Systems)

   null (Ed.)

   Recently there has been a growing interest in fairness-aware recommender systems including fairness in providing consistent performance across different users or groups of users. A recommender system could be considered unfair if the recommendations do not fairly represent the tastes of a certain group of users while other groups receive recommendations that are consistent with their preferences. In this paper, we use a metric called miscalibration for measuring how a recommendation algorithm is responsive to users’ true preferences and we consider how various algorithms may result in different degrees of miscalibration for different users. In particular, we conjecture that popularity bias which is a well-known phenomenon in recommendation is one important factor leading to miscalibration in recommendation. Our experimental results using two real-world datasets show that there is a connection between how different user groups are affected by algorithmic popularity bias and their level of interest in popular items. Moreover, we show that the more a group is affected by the algorithmic popularity bias, the more their recommendations are miscalibrated. 

   more » « less
5. Grid Search Hyperparameter Tuning in Additive Manufacturing Processes

   Ogunsanya, M.; Isichei, J.; Desai, S. (July 2023, Manufacturing letters)

   In Machine learning (ML) and deep learning (DL), hyperparameter tuning is the process of selecting the combination of optimal hyperparameters that give the best performance. Thus, the behavior of some machine learning (ML) and deep learning (DL) algorithms largely depend on their hyperparameters. While there has been a rapid growth in the application of machine learning (ML) and deep learning (DL) algorithms to Additive manufacturing (AM) techniques, little to no attention has been paid to carefully selecting and optimizing the hyperparameters of these algorithms in order to investigate their influence and achieve the best possible model performance. In this work, we demonstrate the effect of a grid search hyperparameter tuning technique on a Multilayer perceptron (MLP) model using datasets obtained from a Fused Filament Fabrication (FFF) AM process. The FFF dataset was extracted from the MakerBot MethodX 3D printer using internet of things (IoT) sensors. Three (3) hyperparameters were considered – the number of neurons in the hidden layer, learning rate, and the number of epochs. In addition, two different train-to-test ratios were considered to investigate their effects on the AM process data. The dataset consisted of five (5) dominant input parameters which include layer thickness, build orientation, extrusion temperature, building temperature, and print speed and three (3) output parameters: dimension accuracy, porosity, and tensile strength. RMSE, and the computational time, CT, were both selected as the hyperparameter performance metrics. The experimental results reveal the optimal configuration of hyperparameters that contributed to the best performance of the MLP model. 

   more » « less