Normalizing flows provide an elegant approach to generative modeling that allows for efficient sampling and exact density evaluation of unknown data distributions. However, current techniques have significant limitations in their expressivity when the data distribution is supported on a lowdimensional manifold or has a non-trivial topology. We introduce a novel statistical framework for learning a mixture of local normalizing flows as “chart maps” over the data manifold. Our framework augments the expressivity of recent approaches while preserving the signature property of normalizing flows, that they admit exact density evaluation. We learn a suitable atlas of charts for the data manifold via a vector quantized autoencoder (VQ-AE) and the distributions over them using a conditional flow. We validate experimentally that our probabilistic framework enables existing approaches to better model data distributions over complex manifolds.
more »
« less
Equivariant Manifold Flows
Tractably modelling distributions over manifolds has long been an important goal in the natural sciences. Recent work has focused on developing general machine learning models to learn such distributions. However, for many applications these distributions must respect manifold symmetries—a trait which most previous models disregard. In this paper, we lay the theoretical foundations for learning symmetry-invariant distributions on arbitrary manifolds via equivariant manifold flows. We demonstrate the utility of our approach by learning quantum field theory-motivated invariant SU(n) densities and by correcting meteor impact dataset bias.
more »
« less
- Award ID(s):
- 2008102
- NSF-PAR ID:
- 10387209
- Editor(s):
- Ranzato, M.; Beygelzimer, A.; Dauphin, Y.; Liang, P.S.; Wortman Vaughan, J.
- Date Published:
- Journal Name:
- Advances in neural information processing systems
- Volume:
- 34
- ISSN:
- 1049-5258
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
We show that all versions of Heegaard Floer homology, link Floer homology, and sutured Floer homology are natural. That is, they assign concrete groups to each based 3-manifold, based link, and balanced sutured manifold, respectively. Furthermore, we functorially assign isomorphisms to (based) diffeomorphisms, and show that this assignment is isotopy invariant. The proof relies on finding a simple generating set for the fundamental group of the “space of Heegaard diagrams,” and then showing that Heegaard Floer homology has no monodromy around these generators. In fact, this allows us to give sufficient conditions for an arbitrary invariant of multi-pointed Heegaard diagrams to descend to a natural invariant of 3-manifolds, links, or sutured manifolds.more » « less
-
Abstract Given a contact structure on a manifold V together with a supporting open book decomposition, Bourgeois gave an explicit construction of a contact structure on $$V \times {\mathbb {T}}^2$$ V × T 2 . We prove that all such structures are universally tight in dimension 5, independent of whether the original contact manifold is itself tight or overtwisted. In arbitrary dimensions, we provide obstructions to the existence of strong symplectic fillings of Bourgeois manifolds. This gives a broad class of new examples of weakly but not strongly fillable contact 5-manifolds, as well as the first examples of weakly but not strongly fillable contact structures in all odd dimensions. These obstructions are particular instances of more general obstructions for $${\mathbb {S}}^1$$ S 1 -invariant contact manifolds. We also obtain a classification result in arbitrary dimensions, namely that the unit cotangent bundle of the n -torus has a unique symplectically aspherical strong filling up to diffeomorphism.more » « less
-
Gaussian processes (GPs) are very widely used for modeling of unknown functions or surfaces in applications ranging from regression to classification to spatial processes. Although there is an increasingly vast literature on applications, methods, theory and algorithms related to GPs, the overwhelming majority of this literature focuses on the case in which the input domain corresponds to a Euclidean space. However, particularly in recent years with the increasing collection of complex data, it is commonly the case that the input domain does not have such a simple form. For example, it is common for the inputs to be restricted to a non-Euclidean manifold, a case which forms the motivation for this article. In particular, we propose a general extrinsic framework for GP modeling on manifolds, which relies on embedding of the manifold into a Euclidean space and then constructing extrinsic kernels for GPs on their images. These extrinsic Gaussian processes (eGPs) are used as prior distributions for unknown functions in Bayesian inferences. Our approach is simple and general, and we show that the eGPs inherit fine theoretical properties from GP models in Euclidean spaces. We consider applications of our models to regression and classification problems with predictors lying in a large class of manifolds, including spheres, planar shape spaces, a space of positive definite matrices, and Grassmannians. Our models can be readily used by practitioners in biological sciences for various regression and classification problems, such as disease diagnosis or detection. Our work is also likely to have impact in spatial statistics when spatial locations are on the sphere or other geometric spaces.more » « less
-
Lu, Jianfeng ; Ward, Rachel (Ed.)The Euclidean scattering transform was introduced nearly a decade ago to improve the mathematical understanding of convolutional neural networks. Inspired by recent interest in geometric deep learning, which aims to generalize convolutional neural networks to manifold and graph-structured domains, we define a geometric scattering transform on manifolds. Similar to the Euclidean scattering transform, the geometric scattering transform is based on a cascade of wavelet filters and pointwise nonlinearities. It is invariant to local isometries and stable to certain types of diffeomorphisms. Empirical results demonstrate its utility on several geometric learning tasks. Our results generalize the deformation stability and local translation invariance of Euclidean scattering, and demonstrate the importance of linking the used filter structures to the underlying geometry of the data.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- The DOI is not currently available.
