More Like this

1. A Picture of the Space of Typical Learnable Tasks

   Ramesh, Rahul; Mao, Jialin; Griniasty, Itay; Yan, Rubing; Teoh, Han Kheng; Transtrum, Mark K; Sethna, James P; Chaudhari, Pratik (May 2023, Proceedings of the 40th International Conference on Machine Learning)

   We develop information geometric techniques to understand the representations learned by deep networks when they are trained on different tasks using supervised, meta-, semi-supervised and con- trastive learning. We shed light on the following phenomena that relate to the structure of the space of tasks: (1) the manifold of probabilistic models trained on different tasks using different represen- tation learning methods is effectively low-dimen- sional; (2) supervised learning on one task results in a surprising amount of progress even on seem- ingly dissimilar tasks; progress on other tasks is larger if the training task has diverse classes; (3) the structure of the space of tasks indicated by our analysis is consistent with parts of the Word- net phylogenetic tree; (4) episodic meta-learning algorithms and supervised learning traverse differ- ent trajectories during training but they fit similar models eventually; (5) contrastive and semi-su- pervised learning methods traverse trajectories similar to those of supervised learning. We use classification tasks constructed from the CIFAR- 10 and Imagenet datasets to study these phenom- ena. Code is available at https://github.com/grasp- lyrl/picture of space of tasks. 

   more » « less
2. Adaptive von Mises–Fisher Likelihood Loss for Supervised Deep Time Series Hashing

   https://doi.org/10.1109/ICMLA66185.2025.00059  

   Perez, Juan Manuel; Garcia, Kevin; Berry, Brooklyn; Song, Dongjin; Gao, Yifeng (December 2025, IEEE)

   Indexing time series by creating compact binary representations is a fundamental task in time series data mining. Recently, deep learning-based hashing methods have proven effective for indexing time series based on semantic meaning rather than just raw similarity. The purpose of deep hashing is to map samples with the same semantic meaning to identical binary hash codes, enabling more efficient search and retrieval. Unlike other supervised representation learning methods, supervised deep hashing requires a discretization step to convert real-valued representations into binary codes, but this can induce significant information loss. In this paper, we propose a von Mises–Fisher (vMF) hashing loss. The proposed deep hashing model maps data to an M-dimensional hyperspherical space to effectively reduce information loss and models each data class as points following distinct vMF distributions. The designed loss aims to maximize the separation between each modeled vMF distribution to provide a better way to maximize the margin between each semantically different data sample. Experimental results show that our method outperforms existing baselines. The implementation is publicly available at https://github.com/jmpq97/vmf-hashing 

   more » « less
3. Learning locality-sensitive bucketing functions

   https://doi.org/10.1093/bioinformatics/btae228  

   Yuan, Xin; Chen, Ke; Li, Xiang; Shi, Qian; Shao, Mingfu (June 2024, Bioinformatics)

   Abstract MotivationMany tasks in sequence analysis ask to identify biologically related sequences in a large set. The edit distance, being a sensible model for both evolution and sequencing error, is widely used in these tasks as a measure. The resulting computational problem—to recognize all pairs of sequences within a small edit distance—turns out to be exceedingly difficult, since the edit distance is known to be notoriously expensive to compute and that all-versus-all comparison is simply not acceptable with millions or billions of sequences. Among many attempts, we recently proposed the locality-sensitive bucketing (LSB) functions to meet this challenge. Formally, a (d1,d2)-LSB function sends sequences into multiple buckets with the guarantee that pairs of sequences of edit distance at most d1 can be found within a same bucket while those of edit distance at least d2 do not share any. LSB functions generalize the locality-sensitive hashing (LSH) functions and admit favorable properties, with a notable highlight being that optimal LSB functions for certain (d1,d2) exist. LSB functions hold the potential of solving above problems optimally, but the existence of LSB functions for more general (d1,d2) remains unclear, let alone constructing them for practical use. ResultsIn this work, we aim to utilize machine learning techniques to train LSB functions. With the development of a novel loss function and insights in the neural network structures that can potentially extend beyond this specific task, we obtained LSB functions that exhibit nearly perfect accuracy for certain (d1,d2), matching our theoretical results, and high accuracy for many others. Comparing to the state-of-the-art LSH method Order Min Hash, the trained LSB functions achieve a 2- to 5-fold improvement on the sensitivity of recognizing similar sequences. An experiment on analyzing erroneous cell barcode data is also included to demonstrate the application of the trained LSB functions. Availability and implementationThe code for the training process and the structure of trained models are freely available at https://github.com/Shao-Group/lsb-learn. 

   more » « less
4. GD-VAEs: Geometric dynamic variational autoencoders for learning nonlinear dynamics and dimension reductions

   https://doi.org/10.1016/j.jcp.2025.114127  

   Lopez, Ryan; Atzberger, Paul J (September 2025, Journal of Computational Physics)

   We develop data-driven methods incorporating geometric and topological information to learn parsimonious representations of nonlinear dynamics from observations. The approaches learn nonlinear state-space models of the dynamics for general manifold latent spaces using training strategies related to Variational Autoencoders (VAEs). Our methods are referred to as Geometric Dynamic (GD) Variational Autoencoders (GD-VAEs). We learn encoders and decoders for the system states and evolution based on deep neural network architectures that include general Multilayer Perceptrons (MLPs), Convolutional Neural Networks (CNNs), and other architectures. Motivated by problems arising in parameterized PDEs and physics, we investigate the performance of our methods on tasks for learning reduced dimensional representations of the nonlinear Burgers Equations, Constrained Mechanical Systems, and spatial fields of Reaction-Diffusion Systems. GD-VAEs provide methods that can be used to obtain representations in manifold latent spaces for diverse learning tasks involving dynamics. 

   more » « less
5. Geometric neural operators (gnps) for data-driven deep learning in non-euclidean settings

   https://doi.org/10.1088/2632-2153/ad8980  

   Quackenbush, B; Atzberger, P J (November 2024, Machine Learning: Science and Technology)

   We introduce Geometric Neural Operators (GNPs) for data-driven deep learning of geometric features for tasks in non-euclidean settings. We present a formulation for accounting for geometric contributions along with practical neural network architectures and factorizations for training. We then demonstrate how GNPs can be used (i) to estimate geometric properties, such as the metric and curvatures of surfaces, (ii) to approximate solutions of geometric partial differential equations on manifolds, and (iii) to solve Bayesian inverse problems for identifying manifold shapes. These results show a few ways GNPs can be used for incorporating the roles of geometry in the data-driven learning of operators. 

   more » « less