More Like this

1. Concept Learners for Few-Shot Learning

   Cao, Kaidi; Brbic, Maria; Leskovec, Jure (January 2021, International Conference on Learning Representations (ICLR))

   null (Ed.)

   Developing algorithms that are able to generalize to a novel task given only a few labeled examples represents a fundamental challenge in closing the gap between machine- and human-level performance. The core of human cognition lies in the structured, reusable concepts that help us to rapidly adapt to new tasks and provide reasoning behind our decisions. However, existing meta-learning methods learn complex representations across prior labeled tasks without imposing any structure on the learned representations. Here we propose COMET, a meta-learning method that improves generalization ability by learning to learn along human-interpretable concept dimensions. Instead of learning a joint unstructured metric space, COMET learns mappings of high-level concepts into semi-structured metric spaces, and effectively combines the outputs of independent concept learners. We evaluate our model on few-shot tasks from diverse domains, including fine-grained image classification, document categorization and cell type annotation on a novel dataset from a biological domain developed in our work. COMET significantly outperforms strong meta-learning baselines, achieving 6–15% relative improvement on the most challenging 1-shot learning tasks, while unlike existing methods providing interpretations behind the model’s predictions. 

   more » « less
2. Concept Learners for Few-Shot Learning

   Cao, Kaidi; Brbic, Maria; Leskovec, Jure. (January 2021, International Conference on Learning Representation (ICLR))

   null (Ed.)

   Developing algorithms that are able to generalize to a novel task given only a few labeled examples represents a fundamental challenge in closing the gap between machine- and human-level performance. The core of human cognition lies in the structured, reusable concepts that help us to rapidly adapt to new tasks and provide reasoning behind our decisions. However, existing meta-learning methods learn complex representations across prior labeled tasks without imposing any structure on the learned representations. Here we propose COMET, a meta-learning method that improves generalization ability by learning to learn along human-interpretable concept dimensions. Instead of learning a joint unstructured metric space, COMET learns mappings of high-level concepts into semi-structured metric spaces, and effectively combines the outputs of independent concept learners. We evaluate our model on few-shot tasks from diverse domains, including fine-grained image classification, document categorization and cell type annotation on a novel dataset from a biological domain developed in our work. COMET significantly outperforms strong meta-learning baselines, achieving 6–15% relative improvement on the most challenging 1-shot learning tasks, while unlike existing methods providing interpretations behind the model’s predictions. 

   more » « less
3. Reduction with Respect to the Effective Order and a New Type of Dimension Polynomials of Difference Modules

   https://doi.org/10.1145/3476446.3535497  

   Levin, Alexander (July 2022, 2022 International Symposium on Symbolic and Algebraic Computation)

   General Chair: Marc Moreno Chair: Lihong Zhi (Ed.)

   We introduce a new type of reduction in a free difference module over a difference field that uses a generalization of the concept of effective order of a difference polynomial. Then we define the concept of a generalized characteristic set of such a module, establish some properties of these characteristic sets and use them to prove the existence, outline a method of computation and find invariants of a dimension polynomial in two variables associated with a finitely generated difference module. As a consequence of these results, we obtain a new type of bivariate dimension polynomials of finitely generated difference field extensions. We also explain the relationship between these dimension polynomials and the concept of Einstein’s strength of a system of difference equations. 

   more » « less
4. A Class of Nonlinear Elasticity Problems with No Local but Many Global Minimizers

   https://doi.org/10.1007/s10659-023-10026-9  

   Grabovsky, Yury; Truskinovsky, Lev (April 2023, Journal of Elasticity)

   We present a class of models of elastic phase transitions with incompatible energy wells in an arbitrary space dimension, where in a hard device an abundance of Lipschitz global minimizers coexists with a complete lack of strong local minimizers. The analysis is based on the proof that every strong local minimizer in a hard device is also a global minimizer which is applicable much beyond the chosen class of models. Along the way we show that a new demonstration of sufficiency for a subclass of affine boundary conditions can be built around a novel nonlinear generalization of the classical Clapeyron theorem. 

   more » « less
5. Double Descent and Other Interpolation Phenomena in GANs

   Luzi, Lorenzo; Dar, Yehuda; Baraniuk, Richard (June 2021, ArXivorg)

   null (Ed.)

   We study overparameterization in generative adversarial networks (GANs) that can interpolate the training data. We show that overparameterization can improve generalization performance and accelerate the training process. We study the generalization error as a function of latent space dimension and identify two main behaviors, depending on the learning setting. First, we show that overparameterized generative models that learn distributions by minimizing a metric or f-divergence do not exhibit double descent in generalization errors; specifically, all the interpolating solutions achieve the same generalization error. Second, we develop a new pseudo-supervised learning approach for GANs where the training utilizes pairs of fabricated (noise) inputs in conjunction with real output samples. Our pseudo-supervised setting exhibits double descent (and in some cases, triple descent) of generalization errors. We combine pseudo-supervision with overparameterization (i.e., overly large latent space dimension) to accelerate training while performing better, or close to, the generalization performance without pseudo-supervision. While our analysis focuses mostly on linear GANs, we also apply important insights for improving generalization of nonlinear, multilayer GANs. 

   more » « less