The enduring legacy of Euclidean geometry underpins classical machine learning, which, for decades, has been primarily developed for data lying in Euclidean space. Yet, modern machine learning increasingly encounters richly structured data that is inherently non-Euclidean. This data can exhibit intricate geometric, topological and algebraic structure: from the geometry of the curvature of space-time, to topologically complex interactions between neurons in the brain, to the algebraic transformations describing symmetries of physical systems. Extracting knowledge from such non-Euclidean data necessitates a broader mathematical perspective. Echoing the 19th-century revolutions that gave rise to non-Euclidean geometry, an emerging line of research is redefining modern machine learning with non-Euclidean structures. Its goal: generalizing classical methods to unconventional data types with geometry, topology, and algebra. In this review, we provide an accessible gateway to this fast-growing field and propose a graphical taxonomy that integrates recent advances into an intuitive unified framework. We subsequently extract insights into current challenges and highlight exciting opportunities for future development in this field.
]]>For nearly two millennia, Euclid's Elements of Geometry formed the backbone of our understanding of space and shape. This 'Euclidean' view of geometry-characterized by flat planes and straight lines-remained unquestioned until the 19th century. Only then, did mathematicians venture "beyond" to develop the principles of non-Euclidean geometry on curved spaces. Their pioneering work revealed that there is no singular geometry. Instead, Euclidean geometry is but one in a mathematical universe of geometries, each of which can be used to illuminate different structures in nature-from the mechanics of celestial bodies embracing the curvature of spacetime to the topologically and algebraically complex electrical patterns of neurons in natural and artificial neural networks. This non-Euclidean revolution was part of a greater trend towards generalization and abstraction in 19th and 20th century mathematics. In addition to expanding the realm of geometry, mathematicians proceeded to define more abstract notions of space, freed from rigid geometric concepts like distances and angles. This gave rise to the field of topology, which examines the properties of a space that are preserved under continuous transformations such as stretching and bending. By abstracting away from the rigidity of geometric structures, topology emphasizes more general spatial properties such as continuity and connectedness. Indeed, two structures that look very different from a geometric perspective may be considered topologically equivalent. The famous of example of this is a donut and coffee mug, which are topologically equivalent since one can be continuously deformed into the other. This notion of abstract equivalence was supported by the simultaneous development of the field of abstract algebra, which examines the symmetries of an object-the transformations that leave its fundamental structure unchanged. These mathematical ideas quickly found applications in the natural sciences, and revolutionized how we model the world.
A similar revolution is now unfolding in machine learning, (see for example Bronstein et al. (2017)). In the last two decades, a burgeoning body of research has expanded the horizons of machine learning, moving beyond the flat, Euclidean spaces traditionally used in data analysis to embrace the rich variety of structures offered by non-Euclidean geometry, topology, and abstract algebra. This movement includes the generalization of classical statistical theory and machine learning in the field of Geometric Statistics (Pennec, 2006;Guigui et al., 2023) as well as deep learning models in the fields of Geometric, (Bronstein et al., 2021), Topological (Hajij et al., 2023;Bodnar, 2022), and Equivariant (Cohen, 2021) Deep Learning. In the 20th century, non-Euclidean geometry radically transformed how we model the world with pen and paper. In the 21st century, it is poised to revolutionize how we model the world with machines.
This review article provides an accessible introduction to the core concepts underlying this movement. We organize the models in this body of literature into a coherent taxonomy defined by the mathematical structure of both the data and the machine learning model. In so doing, we clarify distinctions between approaches and we highlight challenges and highpotential areas of research that are as-of-yet unexplored. We begin by introducing the essential mathematical background in Section II, and turn to an analysis of mathematical structure in data in Section III before introducing our ontology of machine learning and deep learning methods in Section IV and V. We explore the associated landscape of open-source software libraries in Section VII, and delve into the movement's key application domains in Section VIII. Accordingly, this review article reveals how machine learning born from the elegant mathematics of geometry, topology, and algebra has been developed, implemented, and adapted to propose transformative Fig. 1. Beyond Euclid: Discrete Topological Structures. Left: Euclidean space discretized into a regular grid. Right: Discrete topological spaces that go beyond classical discretized Euclidean space. Graphs, Cellular Complexes, Hypergraphs relax the assumption of the regular grid and allow points to be connected with more complex relationships. The arrow +topology indicates the addition of a non-Euclidean, discrete topological structure. Adapted from Papillon et al. (2023). solutions to real-world challenges.
We first provide an accessible and concise introduction to the essential mathematical concepts. For readability, we define the concepts primarily linguistically here, and refer the reader to the works Guigui et al. (2023); Bronstein et al. (2021); Hajij et al. (2023); Cohen (2021) for their precise mathematical definitions.
Topology (shown throughout with purple terms), geometry (shown throughout with orange terms), and algebra (shown throughout with blue terms), are branches of mathematics that study the properties of abstract spaces. In machine learning, data can possess explicit spatial structure-such as an image of a brain scan, or a rendering of a protein surface. Even when data is not overtly spatial, a dataset can be naturally conceptualized as a set of samples drawn from an abstract surface embedded in a high-dimensional space. Understanding the "shape" of data-that is, the shape of the space to which this data belongs-can give important insights into the patterns of relationships that give data its meaning.
Topology, geometry and algebra each provide a different lens and set of tools for studying the properties of data spaces and their "shapes." Topology lends the most abstract, flexible perspective and considers spaces as stretchy structures that can be continuously deformed so long as connectivity and continuity are preserved. Topology thus studies the relationships between points. Geometry allows us to quantify familiar properties such as distances and angles, in other words: perform measurements on points. Algebra provides the tools to study the symmetries of an object-the transformations that can be applied while leaving its fundamental structure invariant.
A topological space is a set of points equipped with a structure known as a topology that establishes which points in the set are "close" to each other. The topology gives spatial structure to an otherwise unstructured set. Formally, a topology is defined as a collection of open sets. An open set is a collection of points in a spatial region that excludes points on the boundary. By grouping points into open sets, we can use terms like 'neighborhoods' and 'paths' to reference "closeness" and other abstract relationships between points. This gives us a way to formalize concepts like continuity (can one travel from one location to another without teleporting?) and connectedness (are two locations in the same neighborhood or region?).
Given the generality of topological structures, topological spaces can be quite exotic. Within this paper, we do not consider continuous topological spaces, and we explicitly restrict the term topology to discrete topological structures, such as graphs, cellular complexes, and hypergraphs. We consider these spaces to be a generalization of the discretized Euclidean space. Indeed, Euclidean space discretizes into a regular grid, while graphs, cellular complexes, and hypergraphs allow for more flexible patterns of connectivity, where points may interact through complex relationships, as shown in Figure 1.
A manifold is a continuous space that locally "resembles" (is homeomorphic to) Euclidean space in the neighborhood of every point. In other words, it is locally linear, and it does not intersect itself. Though locally it resembles flat space, its global shape may exhibit curvature. Without additional structure, the manifold can be seen as a soft and elastic surface. Introducing a notion of distance gives the manifold a Fig. 3. Beyond Euclid: Algebraic Transformations. Left: Euclidean space. Right: Group transformations that act on the elements of an Euclidean space: 2D translation from the group R 2 , 2D rotation from the group SO(2), 2D reflection from the group {1, -1}, and a combination of translation and rotation from the Special Euclidean group SE(2). The arrow +algebra indicates the addition of the non-Euclidean algebraic structure defining a group action. more definite structure, as enforcing how far apart points are constrains its overall geometry.
There are several approaches to defining a distance on the manifold. A powerful method is to use a Riemannian manifold, which can be thought of as a smooth high-dimensional surface that locally resembles flat Euclidean space but may curve globally. To measure distance on such a surface, we use a Riemannian metric. Intuitively, this acts like a local ruler at each point, telling us how to measure lengths of curves that pass through that point. Mathematically, it is defined as a positive definite inner product that varies smoothly on the tangent space at each point. By integrating these local measurements along a curve, we can compute the total distance along that curve. A geodesic generalizes the Euclidean concept of a "straight line" to curved Riemannian manifolds. A Riemannian geodesic is a curve that traces the locally shortest path between two points.
There exist different flavors of Riemannian manifolds, such as the spheres, hyperbolic spaces, and tori. We consider these spaces to be generalizations of the continuous Euclidean space. Indeed, Euclidean spaces are globally flat, while spheres, hyperbolic spaces and tori can exhibit curvature -as illustrated in Figure 2.
A group is a set of elements equipped with a rule for combining them, called a binary operation. This operation must satisfy certain properties: closure, associativity, the existence of an identity element, and the existence of inverses. While a group itself is just a set with these properties, in many contexts, such as ours, the elements of the group can be interpreted as transformations, like 3D rotations. This interpretation arises when we define a group action or representation, where the elements of the group act as operations on a space. Groups can be discrete or continuous. A Lie group is a continuous group, defined as a smooth manifold equipped with a compatible group structure such that the composition of elements on the manifold obeys the group axioms. An example of a Lie group is the set of special orthogonal matrices in R 3×3 under matrix multiplication, which defines the group of 3-dimensional rotations, SO(3). Each matrix is an element of the group and defines rotation at a certain angle. Lie groups are extensively used in physics where they describe symmetries in physical systems.
A group of transformations, such as SO(3)-the group of 3D rotations-may act on another manifold to transform its elements. A group action maps an element in a manifold to a new location, determined by the group element that transforms them. For example, a group action on a Euclidean space can translate, rotate and reflect its elements, as illustrated in Figure 3. In this paper, we use the term algebra to denote that we equip a space with a group action.foot_0
The mathematics of topology, geometry and algebra provide a conceptual framework to categorize the nature of data found in machine learning. We generally encounter two types of data: either data as coordinates in space (Fig. 4)-for example, the coordinate of the position of an object in a 2D space; or data as signals over a space (Fig. 5)-for example, an image seen as a 3D (RGB) signal defined over a 2D space (the spatial location). In each case, the space can either be a Euclidean space or it can be equipped with topological, geometric and algebraic structures such as the ones introduced in Section II.
Understanding the structure of this space provides essential insights into the nature of the data. These insights can be, in turn, crucial for selecting the machine learning model that will be most suitable to extract knowledge from this data. We note the emergence of a line of research which leverages Euclidean architectures for dealing with non-Euclidean data, such as protein folding (Abramson et al., 2024), and refer the reader to Brehmer et al. (2024) for a discussion on the topic. Next, we provide a graphical taxonomy, shown in Figure 5, to categorize the structures of data based on the mathematics of topology, geometry and algebra.
We start by categorizing the mathematical structures of data, when data points are coordinates in space-as shown in Figure 4, and described in detail below. We denote x as a datapoint within a dataset x 1 , ..x i , ..., x N , and drop its subscript i for conciseness. C1: Point in Euclidean space: This category of data points forms the basis of conventional machine learning and deep learning approaches. Card C1 (white) in Figure 5 shows an example of a point in Euclidean space R m , which represents the dimensions of a flower from the Iris dataset (Fisher, 1936), where n represents the number of features studied.
C2: Point in manifold: Adding a non-Euclidean geometry to the coordinate space, card C2 (orange) illustrates a data point on a manifold M , where the manifold is the sphere. For example, this data point can represent the geographic coordinates of a storm event, i.e., its location on the surface of the earth represented as a sphere.
C3: Point in topological space: Card C3 (light purple) considers a data point that resides in a topological space Ω, here corresponding to a node in a graph. An example in this category would be a data point representing an atom in the graph of a molecule, in which edges represent the bounds between the atoms.
The next three data categories add a group action to the data spaces mentioned above.
C4: Point in Euclidean space equipped with group action: Card C4 (blue) displays a data point in a Euclidean space that has been equipped with a group action. Group actions enable us to model and preserve symmetries in the data. For example, a group action on the flower dimensions could be the change of units, from centimeters to millimeters. In this case, the group of interest is the group of scalings R * + . The action of this group does not change the information contained in the data (the flower will still have the same size in the real world), it only changes the way we encode that information. C5: Point in manifold equipped with group action: Card C5 adds the notion of group action to a manifold. An example of such a group action could be a change of coordinate systems on the sphere, changing the origin of longitudes. In this case, the group of interest would be the group of 3D rotations SO(3). Again, the action of this group represents symmetries in the data: the information content is unchanged (a storm will still happen at the same geographical location in the real world) but the way we encode that information has changed.
C6: Point in topological space equipped with group action: Card C6 equips a topological space with the notion of group action. An example of this is a graph of people equipped with an action that can change the way indexing is done on the dataset. The order in which we index people does not change their social relationships, only the way we represent this data in the computer. The group of interest in this case in the group of permutations.
These categories describe the mathematical structure of the data spaces encountered in machine learning. However, even when data points belong to spaces with topological, geometric, or algebraic structures, their computational representations typically take the form of arrays. These data points may thus appear as vectors in a computer's memory, but this is merely a convenience for processing and storage. The underlying mathematical structure is preserved through constraints imposed on the values of these vectors.
We note that a data point on a manifold exists independently of the array with which a computer represents. For example, the data point on the sphere can be represented by a vector of size 3 encoding its Cartesian coordinates, or by a vector of size 2 representing its latitude and longitude. In both cases, the mathematical nature of the data point is unchanged: it still represents a point on a manifold. For instance, when represented as a 3D coordinate vector, a data point on the sphere is constrained to have unit norm (i.e., its Euclidean length equals one), which encodes the geometric properties defining the sphere. More generally, a norm defines a notion of length in a vector space, and can induce a metric that measures distances between points. In the context of 3D rotations, Zhou et al. (2019) provides an experimental analysis of their computational representations and the impact of representation choice on machine learning models.
In many applications, data points are not coordinates in space but rather functions defined over a space, typically assigning a vector in R m ′ to every point in the space R m . Formally, we can write such a function as x : R m → R m ′ where the input space (here, R m ) is called the domain and the output space (here, R m ′ ) is called the codomain. We refer to data of this type as a signal. Elements of the codomain are called the values of the signal x.
Color images provide a clear example of this. The domain is a bounded region of R 2 : for example, limited to the range [0,1920] on the horizontal axis and [0, 1080] on the vertical axis such that each point in the domain represents a pixel location. Each location is assigned a vector in the codomain R 3 , specifying the intensity on each RGB color channel.
Figure 5 introduces several types of data points as functions from a domain to a codomain, together with real-world data examples. Formalizing a signal in this way gives us the flexibility to represent more general classes of signals using non-Euclidean structures. The domain or the codomain (or both) may be any one of the non-Euclidean spaces introduced in the previous section. We present the different options in the four bottom rows of Figure 5 and detail them below. S1: Euclidean signal on Euclidean domain: This represents the most common type of data in classical deep learning, functions as x i : R n → R n ′ , going from points in domain R n to features in codomain R n ′ . A gray-scale image provides a clear example of this. The domain is a bounded region of R 2 , which is discretized into integer steps, i.e., pixels. Typically, this domain is computationally modeled as Z 2 . Each pixel location is assigned a vector in codomain R, which specifies the intensity in grayscale. S2: Manifold-valued signal on Euclidean domain: This represents signals that can be formalized as a function x i : R n → M ′ , which assigns an element of a manifold M ′ to every point in the space. An example of this occurs in diffusion tensor imaging, in which 3D images of the brain are taken, characterized as voxels in R 3 . Attached to each voxel is a 3×3 covariance matrix that describes how water molecules diffuse in the brain -i.e., an element from the manifold of symmetric positive definite matrices SP D(3) (Pennec, 2006). S3: Manifold-valued signal on manifold domain: In this category, both the point coordinates and the features live on manifolds M and M ′ , respectively, i.e., x i : M → M ′ . Air traffic covariance data is a usecase where the domain is the two-sphere S 2 (earth surface) and the features are 2 × S 2 SPD matrices, i.e., elements of the manifold codomain SP D(2). These matrices can encode covariance matrices representing representing different levels of local complexity (Le Brigant and Puechmorel, 2019). S4: Manifold-valued signal on topological domain: Here, the coordinates of the point live in a topological domain Ω and the features live on a manifold M ′ , such that x i : Ω → M ′ . A representative example for this setting is a human pose. Each joint in the body is a feature encoded on the special orthogonal group SO(3), and the domain can be a non-directed graph. S5: Euclidean signal on topological domain: This represents a signal where point coordinates live on a topological domain Ω and the features live in a Euclidean space R n ′ , that is: x i : Ω → R n ′ . Reusing the example of encoding the human pose, each joint can have a feature encoded in R 3 defined over a non directed graph domain. S6: Euclidean signal on manifold domain: This represents the coordinates of the points on a manifold M and the features in Euclidean space R n ′ , i.e., x i : M → R n ′ . An example is a dataset in which each datapoint is a snapshot of the Earth at a given time showing the distribution of temperatures across the globe: the surface temperatures live on the sphere manifold S 2 and the features are in R (image and dataset credits: Atmo, Inc.). S7: Euclidean signal on Euclidean domain equipped with domain action: Card S7 has function, domain, and features that are the same as in Card S1: x i : R n → R n ′ , but the Euclidean domain is now equipped with a group action. For example, the domain R 2 of an image can be equipped with the group action of the wallpaper group P 4, which enables 90°rotations of the image. S8: Euclidean signal on topological domain equipped with domain action: The function here is x i : Ω → R n ′ , where Ω is equipped with a group action. An example of application here is the pose of the human body, where the domain is a undirected graph representing the body joints and the group action on the domain here is the group of permutation matrices P . S9: Euclidean signal on manifold domain equipped with a domain action: Here, the function is x i : M → R n ′ , where the manifold domain M is equipped with a group action. Using the earth surface temperature example previously defined, we can apply a rotation on the domain of temperature domain, in the case of a change of spherical coordinates for instance. In that case, the action group is the group of rotations SO(3). S10: Euclidean signal on Euclidean domain equipped with domain and codomain actions: This represents functions such as x i : R n → R n ′ , where both R n and R n ′ are equipped with group actions. The illustration in the cards shows a vector field in a domain R 2 with features (vectors) in R 2 . In this example, the actions on both the domain and the codomain are from the group of rotations SO(2), which applies the same rotation to each point and for each independent vector in the vector field. S11: Euclidean signal on topological domain equipped with domain and codomain actions: Here, the function is x i : Ω → R n ′ , where both Ω and R n ′ are equipped with group actions. Similarly to the previous example, we can apply a permutation group P on the domain and the action of the rotation group SO(3) on the features. Another relevant example are geometric molecular graphs which also exhibit permutation and Euclidean symmetry. S12: Euclidean signal on manifold domain equipped with domain and codomain actions: Here x
where both M and R n ′ are equipped with group actions.
We present an example of a vector field defined over the sphere S 2 with vectors in R n ′ . Applying a the same group action -an SO(3) rotation -rotates both the coordinates of the vectors and their direction. This can be useful for changes of coordinates.
As in the previous subsection, we emphasize the difference between the mathematical structure of data and the computational representation of data. Until now, we described data as signals represented as functions x over a domain. However, most of the time, the functions x are discretized in the computer. For example, the domain of a function representing a 2D color image as the signal x : R 2 → R 3 is discretized into a grid with p 2 pixels, where each pixel has a value in R 3 . The data point x is therefore represented, in the computer, by an array of length p 2 * 3. A recent line of research, the literature of operators and neural operators (Kovachki et al., 2023), offers a different approach by treating each data point x as a continuous function without discretization.
Remark: Opportunities:
The cases of manifold-valued signals on manifold domain, topological-valued signals on topological domains, or topological-valued signals on manifold domains are not included in the table, since they have rarely been considered in the machine learning and deep learning literatures. These classes represent an avenue for future research. Remark: Data as Spaces: Beyond data as coordinates and data as signals, a third class can be considered: data as spaces, where a data point x is itself a manifold or topological space. For example, in a molecular dataset, each molecule x can be represented as a graph capturing its structure, with atoms as nodes and bonds as edges; the dataset is then a collection of graphs. Similarly, in datasets of 3D scans, such as heart surfaces, each x is a distinct manifold. However, in most cases, x carries additional structure beyond its topology. A molecule, for instance, is not only defined by atomic bonds but also by the 3D positions of its atoms-thus represented as a signal from a graph Ω to R 3 , as in Card S5. Likewise, heart surfaces are described by 3D coordinates at each point, making each x a signal from a manifold M to R 3 . Consequently, these examples ultimately fall into the category of data as signals.
This review uses the two main classes of data representations as coordinates, and as signals, to describe non-Euclidean structure in machine learning and deep learning.
We now review a large and disparate body of literature of non-Euclidean generalizations of algorithms classically defined for data residing in Euclidean spaces. The generalization of machine learning methods to non-Euclidean data first relies on the generalization of their mathematical foundations-probability and statistics-to non-Euclidean spaces. We refer the reader to Pennec (2006) for theoretical foundations on manifolds, and to Pennec et al. (2019) for real-world applications. Beyond probability and statistics, the machine learning algorithms requires non-trivial algorithmic innovations. Such generalizations comprise the bulk of the work reviewed here.
However, we note that there are two "simple" approaches to generalizing Euclidean models that do not require significant algorithmic innovation. They do not work for all algorithms, and they have limitations. Yet, when possible, they have the benefit of implementational simplicity. We briefly describe two classes of such approaches here-what we call "plug-in" methods and tangent space methods.
Non-Euclidean Probability and Statistics.
In a non-Euclidean space, many essential mathematical concepts must be modified to respect the inherent structure of the space. Consider, for example, a dataset consisting of points that lie on the surface of the sphere-for example, the coordinates of different cities around the globe. The Euclidean mean of the points is a value that lies off-manifold-a point lying somewhere "inside" the sphere but not on the sphere. To find the centroid of these points on the manifold, we must instead use the Fréchet mean, pictured in Fig. 6. This is defined as the point that minimizes the sum of squared geodesic distances to all other points in the dataset. By using geodesics, the Fréchet mean is constrained to the manifold, and results in a natural generalization of the notion of a mean to non-Euclidean space. The field of Geometric Statistics defines such non-Euclidean generalizations. We refer the reader to Pennec (2006) for theoretical foundations on manifolds, to Guigui et al. (2023) for a comprehensive introduction to these foundations and to Pennec et al. (2019) for real-world applications.
"Plug-In" Methods: The most straightforward way to generalize machine learning methods to non-Euclidean spaces is to simply replace the definitions of addition, subtraction, distances, and means employed in the Euclidean method with their non-Euclidean counterparts. For example, the knearest-neighbors algorithm can be naturally defined for arbitrary non-Euclidean manifolds by replacing Euclidean distance with geodesics. Similarly, k-means can be generalized using geodesic distances and the Fréchet mean. Any kernel method that makes use of geodesic distance in the kernel function falls into this category as well. Many popular implementations of machine learning algorithms-for example in the scikit-learn package-permit the user specification of a distance function, thus facilitating easy generalization to non-Euclidean spaces. However, many approaches require deeper modifications that go beyond replacing the operations of the standard method. Spherical CNNs (Cohen et al., 2018;Kondor et al., 2018), for instance, cannot apply standard convolutions directly, as translations are not well defined on the sphere. Instead, they define convolution by applying group convolutions over the rotation group SO(3) using spherical harmonics, ensuring rotational equivariance and respecting the sphere's geometry.
Tangent Space Methods: An alternative approach is particularly convenient for non-Euclidean spaces that are manifolds. We call it the tangent space method. It consists in projecting the data from the manifold into the tangent space of a particular point on the manifold, using the so-called exponential map. Importantly, the tangent space can be seen as a Euclidean space. Once the data are mapped to Euclidean space, traditional Euclidean machine learning can be applied. This approach typically achieves better results than applying Euclidean methods directly to the original non-Euclidean data.
However, in many cases-particularly for manifolds with greater curvature-it induces biases in the results due to errors in the local Euclidean approximation of the manifold. Nonetheless, the approach is relatively simple, and can be worthwhile for data lying on manifolds that are flat at the scale of the spread of the data.
Both the plug-in and tangent space methods have the advantage of implementational simplicity. However, many machine learning algorithms require more than just the specification of distances and means, which limits the applicability of the plugin method. Additionally, in many cases, the biases induced by the tangent space projection are intolerable for the application, which limits its scope. Consequently, in many scenarios, it is necessary to explicitly constrain aspects of the algorithms to the manifolds of interest. This comprises the bulk of the work of Non-Euclidean Machine Learning, which we cover in this section.
We note that the methods reviewed in this paper assume that certain topological, algebraic, or geometric structures are known a priori to be present in the data or learning problem. These methods thus require pre-specifying these structures. Sometimes, however, the underlying structure is unknown. A class of methods aims to discover and characterize unknown non-Euclidean structure in data. This class includes topological data analysis, certain manifold learning approaches that learn parameters of a latent manifold, such as in metric learning, and algebraic methods for discovering latent group structure, also known as group learning. Other notable approaches include Noether Networks (Alet et al., 2021), which enforce learned conservation laws, and graph rewiring methods (Topping et al., 2022), which dynamically adapt graph connectivity. These methods are out of scope, as they are used "prior to" non-Euclidean machine learning.
We first introduce our taxonomy defined by the mathematical structure of both the data spaces and the regression model. Then, we review the literature on regression. A complete visual representation of the literature is available in Figure 7.
Taxonomy: In machine learning, regression can be defined as learning a function f going from an input space X to an output space Y . Figure 7 organizes regression models into a taxonomy based on the geometric properties of input and output spaces-see first two columns in Figure 7. Here, we distinguish conventional Euclidean spaces from more complex manifold spaces, setting the stage for a detailed exploration of five key configurations: Euclidean to Euclidean, onedimensional Euclidean to Manifold, Euclidean to Manifold, Manifold to Euclidean, and Manifold to Manifold.
Each configuration is then distinguished by its regression model-see third column in Figure 7. Linear methods assume a relationship between the input variable x ∈ X and output variable y ∈ Y that can be represented by a linear function as y = Ax + b, where A is a matrix of coefficients, and b is a vector of constants-hence constraining the ys to belong to a linear space characterized by the parameters A, b. We note that linear regressions are not appropriate on manifolds since the addition operation, a linear operation, is not well-defined on a manifold, a non-linear space. Consequently, linear methods become geodesic methods on manifolds on rows 2 and 3 of Figure 7, where the relationship between x and y can be represented by a geodesic characterized by a set of parameters analogous to A, b above. Next, non-linear parametric and nongeodesic parametric methods involve relationships described by a fixed set of parameters but in a more complex form, such as polynomials or other non-linear equations y = f θ (x) where θ represents the parameters. Non-parametric approaches, on the other hand, do not assume a parametric form for the relationship between x and y, providing flexibility to model relationships that are directly derived from data. The term nonparametric does not necessarily mean that such models completely lack parameters but that the number and nature of the parameters are flexible and not fixed in advance, contrarily to parametric approaches. Further, these methods are distinguished as Bayesian or frequentist-as denoted by the italicized models in Figure 7. Bayesian methods integrate prior probabilistic distributions with observed data, facilitating the updating of knowledge about the parameters in light of new evidence. This approach contrasts with the remaining methods, which rely exclusively on observed data, typically employing so-called frequentist statistical principles without incorporating prior distributions on parameters.
In what follows, we review regression models according to this geometric taxonomy, row by row in Figure 7. For each category of regression models, we showcase one emblematic, category-defining, paper that reflects the transition from traditional Euclidean analysis to the manifold paradigm. We consider the following papers to be out of scope: 1) Papers that generalize a class of curves (e.g., splines) on manifolds, but do not leverage this generalization to perform regression, 2) Papers that perform interpolation on manifolds, but not regression, 3) Papers whose regression method has been developed for only one type of manifold (e.g., only for spheres).
1) Euclidean Input, Euclidean Output: We review classical regression models that have been generalized to manifolds, in the first row of Figure 7. Initiated with linear regression by Legendre and Gauss (Legendre, 1806), this category has expanded to include nonlinear parametric models, particularly polynomial models introduced by Gergonne (Gergonne, 1815). The inclusion of non-parametric methods is marked by the Nadaraya-Watson kernel methods (Nadaraya, 1964), further local linear models (Fan, 1993), and Breiman's development of random forests (Breiman, 2001). In Bayesian analysis, the linear and polynomial approaches are respectively represented by Bayesian Multilinear (Box and Tiao, 1968) and Polynomial Bayesian models (Halpern, 1973), with the Gaussian Process (Williams and Rasmussen, 1995) illustrating the non-parametric Bayesian perspective.
2) One-dimensional Euclidean Input, Manifold Output:
The earlier generalizations of classical regression models to manifolds involve generalizing the output space. Many of these models consider a one-dimensional input x, and are shown in the second row of Figure 7. Geodesic regression, a manifold generalization of linear models introduced nearly 200 years later, handles one-dimensional Euclidean inputs (Fletcher, 2011). Polynomial regressions on manifolds (Hinkle et al., 2012a) and Bezier-splines fitting on manifolds (Hanik et al., 2020) generalize their Euclidean counterparts to manifolds in the output space, 197 years and 35 years later respectively. Non-parametric methods with output values on manifolds include Fréchetcasted Nadaraya-Watson kernel regression (Davis et al., 2010), and local geodesic regression (Schötz, 2022)-the latter being the counterpart of local linear regression to manifolds. In terms of Bayesian methods, the geodesic regression model was made Bayesian in (Zhang et al., 2020), while manifold polynomial regression turned Bayesian in Muralidharan et al. (2017). Lastly, one nongeodesic, non-parametric, Bayesian model belongs to this category: kernel regressions by Devito and Wang's Brownian motion model (Wang, 2015) which generalizes Nadaraya-Watson's approach 51 years later.
3) Euclidean Input, Manifold Output: Next, we review regression models with outputs on a manifold, for which the input is not restricted to be one-dimensional, in the third row of Figure 7. The Fréchet regression (Petersen and Muller, 2016) generalizes the geodesic regression to higher-dimensional Euclidean inputs. We also find several nongeodesic parametric regression models. One of the earliest works in this category deals with the parametric regression of regularized manifold valued functions (Pennec et al., 2006). A key idea is to rephrase convolutions as weighted Fréchet means of manifold variables, which become the parameters of the implicit function. Also in the nongeodesic parametric regression category, we find the semi-parametric intrinsic regression model by (Shi et al., 2009), or the stochastic development regression by Kühnel and Sommer (2017). Non-parametric methods with manifold-valued outputs include the manifold random forests (Tsagkrasoulis and Montana, 2018) that generalize their Euclidean counterpart 15 years later, the local Fréchet regression (Petersen and Muller, 2016), another multi-dimensional generalization of the local linear regression, and the local extrinsic regression by (Lin et al., 2017). In terms of Bayesian approaches, we observe a lack of methods within geodesic and nongeodesic parametric models. However, we find Bayesian nonparametric methods, such as Manifold Gaussian processes (Yang and Dunson, 2016) and Mallasto's wrapped Gaussian processes (Mallasto and Feragen, 2018) which generalizes Williams and Rasmussen's Gaussian processes 9 and 13 years later.
4) Manifold Input, Euclidean Output: We now review geometric generalizations of Euclidean regression models that have received less attention: models in which the regression input space is a manifold. We start with methods for which the output space is a Euclidean space, in the fourth row of Figure 7. Johnson and Wehrly defined a regression model for an angular (one-dimensional) variable with a Euclidean scalar (one-dimensional) output variable (Johnson and Wehrly, 1978). Chernov's circular regression outputs to higher-dimensional Euclidean spaces, typically two-or three-dimensional (Chernov, 2010). Pelletier's non-parametric regression approach stands out for estimating functions from manifold inputs to Euclidean outputs (Pelletier, 2006). Both methods are non-Bayesian.
In terms of Bayesian methods, we find the Bayesian circular-linear regression method (Gill and Hangartner, 2010), and the Bayesian non-parametric method Matérn Gaussian Processes on manifolds (Borovitskiy et al., 2020).
5) Manifold Input, Manifold Output: The fifth row of Figure 7 presents the most general case of regression, from a geometric perspective. In these models, both the regression input and output spaces are manifolds. This category includes a method that first transforms both input and output data from manifolds to Euclidean spaces, and then apply an auxiliary classical regression model on Euclidean spaces (Guo et al., 2019). In the nonparametric methods, we find Steinke's regular splines methods (Steinke et al., 2008) and Banerjee's manifold kernel regression (Banerjee et al., 2015) generalizing in 2015 both the classical kernel regression from 1964 and its generalization to manifold output space from 2010. At the time of the review, there is no method in this category that adopts the Bayesian point of view.
We first introduce our taxonomy defined by the mathematical structure of both the spaces and the latent embedding model. Then, we review the literature on latent embeddings in Figure 8, with details in the text.
Taxonomy: We define the problem of latent embeddings as transforming (typically, high-dimensional) data from their data space X into a latent space Y (typically, of lower dimension, i.e., dim(Y ) < dim(X)). Figure 8 organizes latent embeddings methods into a taxonomy first based on the geometric properties of the data and latent spaces-see the first two columns. It differentiates between conventional Euclidean spaces and the more complex manifold spaces, setting the stage for the following four key configurations: Euclidean Data to Euclidean Latents, Manifold Data to Euclidean Latents, Euclidean Data to Manifold Latents, and Manifold Data to Manifold Latents. For each configuration, the (usually, lower dimensional) latent space Y is schematically represented as a space of dimension 1: a line for Euclidean spaces, and a circle for manifolds. The (usually, higher-dimensional) data space X is schematically represented as a space of dimension 2: a plane for Euclidean spaces, and a sphere for manifolds. Yet, we emphasize that we review all latent embedding methods here; not only the methods going from dimension 2 to dimension 1. Second, approaches are organized depending on whether they leverage an encoder E, where E is a function mapping data x's to latents ys: E(x) = y. Indeed, while every method converts data into latents, only some methods do so through an explicit encoder function E. Others might only compute the latent y i corresponding to a data point x i through the result of an optimization min y Cost(x i ). The presence of an encoder is represented by a black arrow and the legend E, and the arrow is a dotted arrow if the encoder is non-parametric. The use of optimization is represented by the ! and no black arrow.
Third, approaches are categorized based on how they compute uncertainty on the decoder parameters, i.e., whether they are Bayesian or not. For example, traditional PCA does not incorporate uncertainty, but Bayesian PCA does. Bayesian approaches are distinguished by text in italics.
We further explain if the data is assumed to come from a generative model. A generative model explains how data points are generated from latents using probability distributions. For example, principal component analysis (PCA) has a decoder, but no generative model; while probabilistic PCA uses a decoder with a generative model. We also note if approaches compute an uncertainty on the latents y associated with the data points x. Uncertainty on the latents is represented by the posterior distribution p(y|x).
We now survey the various categories of latent embeddings row by row in Figure 8.
1) Euclidean Data, Euclidean Latents: We describe approaches from this configuration subcolumn by subcolumn. Principal Component Analysis (PCA) (Pearson, 1901) learns a linear subspace, while Probabilistic PCA (PPCA) (Tipping and Bishop, 1999) achieves the same goal within a probabilistic framework relying on a latent variable generative model, which provides a posterior distribution on the latents. Bayesian PCA (Bishop, 1998) additionally learns a probability distribution on the parameters of the models, representing the parameters of the linear subspace (e.g., slope and intercept). These methods are restricted in the type of subspace that can be fitted to the data: only linear subspaces. To lift this restriction, we also find numerous methods that learn nonlinear manifolds from Euclidean data. The whole field of manifold learning fits in this case, and can be subdivided into further subcategories depending on how the learned manifold is being represented: by a nonlinear parametric decoder, by a nonparametric decoder, or without any decoder at all. Here, we only introduce one category-defining method for each of the subcategories we consider.
In the nonlinear parametric decoder category, we introduce autoencoders. Autoencoders (AEs) (Rumelhart et al., 1986) learn a nonlinear subspace of a Euclidean space, while variational autoencoders (VAEs) (Kingma and Welling, 2014) achieve the same goal with a probabilistic framework relying on a latent variable generative model. The Full-VAE model proposed in (Kingma and Welling, 2014) additionally learns a posterior on the pa-rameters of the model, i.e., the parameters of the decoder. The methods in this category all leverage an encoder.
In the nonparametric decoder category, we introduce the generative model principal curves (Hastie and Stuetzle, 1989), which fit a nonlinear manifold to the data, but do not leverage a posterior on the latents. In this same category, we also introduce Local Linear Embedding (LLE) (Roweis and Saul, 2000) and Gaussian Process Latent Variable Models (GP LVM) (Lawrence, 2003), which all provide posteriors on the latents. A probabilistic approach to principal curves (PPS) developed in Chang and Ghosh (2001) also falls under this category. Finally, we introduce the Bayesian GP LVM, which is generative and additionally provides a posterior on the model's parameters. We note that the methods of this category do not leverage any encoder, and instead compute the latent associated with a given data point by solving an optimization problem.
These techniques are based on vector space operations that make them unsuitable for data on manifolds. Consequently, researchers have developed methods for manifold data, which take into account the geometric structure; see next row in the Table, described in the next paragraph.
2) Manifold Data, Euclidean Latents: We describe approaches subcolumn by subcolumn. In the geodesic decoder category, Principal Geodesic Analysis (PGA) (Fletcher et al., 2004;Sommer et al., 2014), tangent PGA (tPGA) (Fletcher et al., 2004), and Geodesic Principal Component Analysis (GPCA) (Huckemann et al., 2010) learn variants of geodesic subspaces, generalizing the concept of a linear subspace to manifolds. As such, these methods represent different generalizations of PCA to manifolds. Probabilistic PGA (Zhang and Fletcher, 2013) achieves the same goal, while adding a latent variable model generating data on a manifold, and hence generalizes probabilistic PCA, 16 years later. Similarly, Bayesian PGA (Zhang and Fletcher, 2013) generalizes Bayesian PCA by including the posterior distribution of the parameters defining the submanifolds, i.e., the base point and tangent vectors defining the principal (geodesic) components. However, these methods are restricted in the type of submanifold that can be fitted to the data, that is: geodesic subspaces at a point.
The restriction to globally defined subspaces based on geodesics can be considered both a strength and a weakness. While it protects from the problem of overfitting with a submanifold that is too flexible, it also prevents the method from capturing possibly nonlinear effects. With current dataset sizes exploding (even within biomedical imaging datasets which have been historically much smaller), the investigation of flexible submanifold learning techniques may become increasingly important.
In the nongeodesic parametric decoder category, we consider various generative models. A natural extension with one-dimensional latents is to define splines using higher order polynomials that are then fitted to a set of points on a Riemannian manifold (Machado and Silva Leite, 2006;Machado et al., 2010) or on a Lie group (Gay-Balmaz et al., 2011). We then consider higher dimensional Euclidean latents. Variational autoencoders have been generalized to manifold data in (Miolane and Holmes, 2020), a methodology that can be applied to both AEs, VAEs and Full-VAEs on manifolds. The AEs, VAEs and Full-VAEs are the only methods that learn a multidimensional nongeodesic submanifold parameterized with a latent variable model.
In the nongeodesic, nonparametric decoder category with one-dimensional Euclidean latents, principal flows (Panaretos et al., 2014) and Riemannian principal curves (Hauberg, 2016) generalize traditional Euclidean principal curves to Riemannian manifolds, 25 years later. The probabilistic Riemannian principal curves (Kang and Oh, 2024) further introduce a probabilistic framework relying on a latent variable generative model, hence generalizing the Euclidean probabilistic principal curves, 23 years later. Lastly, for higher dimensional Euclidean latents: the Riemannian LLE (Maignant et al., 2023), a generative model which generalizes its Euclidean counterpart, the LLE, 23 years later. We note the absence of works performing latent embedding via a nonparametric decoder in a generative model nor in a Bayesian framework. This represents a possible avenue for research.
3) Euclidean Data, Manifold Latents. We describe approaches subcolumn by subcolumn. We first find approaches that belong to the VAE framework, where the latent space is a manifold, even though the data belong to a Euclidean space. For example, the hypersphere VAE by Davidson et al. (2018) proposes a hyperspherical latent space, Falorsi et al. (2018) propose a Lie group latent space, and Mikulski and Duda (2019) propose a toroidal latent space. All of these are generative approaches which provide an (approximate, amortized) posterior on the latent variables, represented as a probability distribution on the manifold of interest. In some sense, these approaches represent the counterpart of Miolane and Holmes (2020) which considers a manifold data space and a Euclidean latent space. Next, we find nonparametric decoder approaches, such as the manifold Gaussian Process Latent Variable Model (GPLVM) (Jensen et al., 2020) which generalizes the GPLVM from the Euclidean case (Lawrence, 2003).
4) No Decoder. Few models avoid using a decoder. For this reason, we briefly survey these models across all geometric data and latent structures. In the Euclidean data with Euclidean latent case (corresponding to row 1), we find Uniform Manifold Approximation and Projection (UMAP) (McInnes et al., 2018) and Isomap (Tenenbaum et al., 2000). These approaches learn lower-dimensional representations of data but do not provide a latent variable generative model, nor a parameterization of the recovered subspace. In the manifold data with Euclidean latent case (corresponding to row 3), a flexible generalization of linear subspaces to manifolds is given by barycentric subspaces (Pennec, 2018), where the submanifold is defined implicitly through geodesics to several reference points. Iterated Frame Bundle Development (IFBD) (Sommer, 2013) is another optimization method that iteratively builds principal coordinates along new directions. GEO-MANCER (Pfau et al., 2020) also falls into this category: it discovers disentangled latent factors by analyzing how local tangent spaces transform under parallel transport. This non-parametric method constructs subspace-valued features via spectral diffusion geometry, revealing latent product structure. In the Euclidean data with manifold latent case (row 4), we first embed data points in X into a hyperbolic latent space Y using Poincaré embeddings Nickel and Kiela (2017). Hyperbolic space is useful for representing data with hierarchical structure. Second, the Riemannian SNE (Bergsson and Hauberg, 2024) generalizes traditional Euclidean SNE (van der Maaten and Hinton, 2008) but 14 years later. Finally, the last decoder-free model, called principal subbundles (Akhøj et al., 2023), is the only model designed for manifold data and manifold latents. Indeed, we note that there are no decoder-based approaches for latent embeddings for this case. Overall, the sparsity of decoder-free latent embedding models leaves open opportunities for many choices of geometric data/latents.
Taxonomy: To complement the geometric view of Figure 7, Figure 9 classifies regression models whose input space X, output space Y , or both are topological structures. This is specified in the first two columns of both tables. Here, we distinguish conventional Euclidean spaces from more complex topological spaces, setting the stage for a detailed exploration of four configurations: Euclidean to Graph, Graph to Euclidean, Hypergraph to Euclidean, Cell Complex to Euclidean.
For each configuration the figure further distinguishes labeled complexes, where node-, edge-or cell-level features are provided, from unlabeled ones that encode only incidence information, by dividing each input/output pair into two rows.
When features are present we specify in the text the exact level(s) on which they reside.
The configurations are similarly distinguished by their regression models and separated by whether they are parametric or non-parametric, see third column of both tables. Unlike Figure 7, the topology table omits a Linear/Geodesic column because concepts such as straight lines or geodesics require a metric structure that generic topological objects (e.g., graphs, hypergraphs, or complexes) do not possess.
Further, these methods are distinguished as Bayesian or frequentist-as denoted by the italicized works in Figure 9.
It is important to note that depending on the task, the regressor may output the entire topological object, such as a full graph, hypergraph, or complex, or more granular elements like nodeor edge-level predictions. The accompanying text specifies the level of the output of the regression models.
In what follows, we review regression models according to this topological taxonomy, row by row in Figure 9. We consider the following papers to be out of scope: Papers that perform interpolation or classification on topological structures, but not regression.
1) Euclidean Input, Graph Output: A first class of methods predicts the structure of entire graphs, either using parametric models (Calissano et al., 2022) or nonparametric alternatives (Severn et al., 2021;Zhou and Müller, 2022). When the predicted graph includes nodelevel features, Bayesian graphical regression offers a probabilistic framework (Ni et al., 2018). Shifting focus from entire graphs to individual nodes, a distinct line of work assumes the graph structure is known and uses it as a regularizer to predict node-level features (Kovac and Smith, 2011;Venkitaraman et al., 2019).
2) Graph Input, Euclidean Output: When predicting from graphs with features, parametric methods such as graph neural networks arise -see Section V. Non-parametric approaches decompose the graph into subgraphs and apply various regression models (Tsuda, 2007;Saigo et al., 2009Saigo et al., , 2008;;Fei and Huan, 2009). We note that, even for unfeatured graphs, one can define basic node features-such as constant values, node degrees, or structural properties-to enable feature-based prediction. Last, a distinct line of work predicts at the node level, i.e, the graph is known and is used as a regularizer of the regression in (Jochmans and Weidner, 2019).
3) Hypergraph Input, Euclidean Output: As with graphs, the regression can predict from featured hypergraphs. Parametric models, including hypergraph neural networks, address this task (see Section V), while non-parametric methods remain unexplored, offering a promising direction for future work. Even without features, structural properties of the hypergraph can be used to define them. Finally, a distinct class of methods uses a single node feature as input of the regression, using the hypergraph as a regularizer (Zhu et al., 2019).
4) Cell Complex Input, Euclidean Output: The regression can also predict from featured simplicial and cellular complexes, with parametric models including simplicial and cellular neural networks (see Section V) and nonparametric methods unexplored. A method using a single node feature as input of the regression is presented in a Bayesian framework in (Alain et al., 2024).
Taxonomy: In this setting, we note that the embedding approach relocates data between Euclidean and topological domains without necessarily lowering dimension-changing the representation rather than necessarily compressing it.
Figure 10 therefore classifies latent embedding methods first by the nature of the data and latent spaces, see the first two columns, covering six source-target configurations: Point Cloud to Graph, Point Cloud to Simplicial or Cellular Complex, Point Cloud to Hypergraph, Graph to Euclidean, Simplicial or Cellular Complex to Euclidean, and Hypergraph to Growing Neural Gas Network [Fritzke94] AMES [Lu23] Neural Relational Inference [Kipf18] Graph Learning CN [Jiang19] DGM for GCN [Kazi23] High-dim Lasso Graph Selection [Meinshausen06] Learning Discrete Structures for GNN [Franceschi19] Relative Neighbourhood Graph [Toussaint80] Adaptive Hypergraph Learning [Yu12] Clique-Hypergraph Clustering [Hu14] L1-Hypergraph Modeling [Wang15] Elastic Net Hypergraph [Lui17] Higher Connectivity of Compact Spaces [Vietoris27] 3D alpha shapes [Edelsbrunner94] Estimation Witness Complexes [Silva04] TDA & 3D Recognition [Singh07] Reconstruction Witness Complexes [Guibas07] Fast construction Vietoris-Rips complex [Zomorodian10] Efficient construction Čech complex [Dantchev12] Simplicial Complex Reconstruction [Wang22] Simplicial-Complex Tumor Unmixing [Roman15] Mixture of Gaussians + MST Lifting [Telyatnikov24] PointNet++ Lifting [Bernardez24] Voronoi Lifting [Bernardez 24] Alpha Complex Lifting [Bernardez 24] Random Flag Complex Lifting [Bernardez24] [S PPCA [Li09] Hypergraph 3D Pose AE [Hong16] Molecular Hypergraph [Kajino19] Heterogeneous Hypergraph VAE [Fan22] Heterogeneous Hypergraph VAE [Fan22] Hypergraph Learning [Zhou06] GP on Hypergraphs [Pinder21] Latent Hypergraph Space Modeling [Turnbull23] Generative Hypergraph Spectral Embedding [Gong23] Hypergraph Learning & Embedding [Zhou06] Random Walks on Simplicial Complexes [Schaub19] Simplex2Vec [Billings19] k-simplex2vec [Hacker20] Random Walks on Simplicial Complexes [Schaub19] Cell Complex AE [Hajij20] Simplicial Complex AE [Hajij22] Cell Complex AE [Hajij20] Simplicial Complex AE [Hajij22] Simplicial Representation with Neural -Form [Maggs24] Euclidean. We do not include Euclidean to Euclidean in this table because that is covered in Figure 8. The review in (Hensel et al., 2021) offers additional details.
For each configuration the figure further distinguishes labeled complexes, where node-, edge-or cell-level features are provided, from unlabeled ones that encode only incidence information, by dividing each input/output pair into two rows. When features are present, we specify in the text the exact levels (node, edges, etc) on which they reside. Models that can operate with or without features are included in both rows. Each configuration is then distinguished by the approach to latent embedding-see third column in Figure 10, depending on whether they are parametric or non-parametric. We do not include a Linear column for the same reasons as in the previous section.
We once again note that depending on the task, the latent embedding model may input or embed the entire topological object, such as a full graph, hypergraph, or complex, or more granular elements like nodes, edges or cells. The accompanying text specifies the level of inputs or embeddings for each model.
We now survey the various categories of latent embeddings row by row in Figure 10. We consider the following "lifting" methods to be out of scope: methods that go from a graph to a topological domain, or between topological domains, and refer to (Telyatnikov et al., 2024) for their review.
1) Point Cloud Input, Graph Output: We describe approaches from this configuration starting with unlabeled graphs and then moving onto labeled. Within the unlabeled graphs, the majority of latent embedding approaches are node level, meaning that each point in the point cloud becomes a node in the outputted graph. Node level parametric approaches include graph construction with Lasso regression (Meinshausen and Bühlmann, 2006) bilevel Bernoulli-edge learning (LDS) and (Franceschi et al., 2019). For nonparametric unlabeled approaches, we have the node level approach Toussaint (1980).
For labeled outputs, parametric node level approaches include the neural relational inference model Kipf et al. (2018) that utilizes a variational GNN and learns features on edges, Jiang et al. (2019a) which jointly learns an adaptive adjacency via a trainable similarity kernel inside a graph-learning-convolutional network and features on nodes, and Kazi et al. ( 2023) which inserts a
Gumbel-Softmax edge-sampling module to learn taskspecific graph structure end-to-end and features on nodes. Nonparametric node level labeled approaches include the growing neural gas network which outputs a graph with node and edge features (Fritzke, 1994) and Attentional Multi-Embedding Selection (Lu et al., 2023).
2) Point Cloud Input, Simplicial/Cellular Complex Output: For unlabeled parametric models, reconstruction of simplicial complexes from binary contagion and Ising data constructs a simplicial complex where points become the 0-simplices of the complex (Wang et al., 2022).
No unlabeled nonparametric methods exist, indicating an opportunity for further exploration.
In terms of labeled parametric models, Roman et al. ( 2015) is a component level approach that clusters the point cloud, then fits a simplex to each cluster and treats the simplex vertices as the genomic profiles of unobserved cell types. For non parametric models, nodelevel the Vietoris-Rips complex (Vietoris, 1927) outputs a simplicial complex with original features on nodes and no features on edges nor simplices. An efficient approach to this construction is presented in (Zomorodian, 2010). Edelsbrunner and Mücke (1994) outputs a sequence of simplicial complices, with the original features on the nodes. Dantchev and Ivrissimtzis (2012) does the same but outputs a single simplicial complex. Similarly, Silva and Carlsson (2004) and Guibas and Oudot (2007) have the same structure for a single complex but with only a subset of the original features. Original points are clustered and then those clusters become a 0-simplex in the complex in (Singh et al., 2007).
This selection is not exhaustive since all of topological data analysis and persistence homology would fall here.
3) Point Cloud Input, Hypergraph Output: Beginning with unlabeled outputs, no methods exist-indicating an opportunity for further exploration. For labeled outputs, a parametric method is the Mixture of Gaussians MST lifting, transforming a point cloud into a hypergraph by employing a Gaussian mixture model and constructing a minimal spanning tree between the generated Gaussians Telyatnikov et al. (2024). Nonparametric methods include Yu et al. (2012), Hu et al. (2014), Wang et al. (2015) and Liu et al. (2017), which are node-level and every inputted point becomes a hypergraph vertex, and hyperedges receive learned weights. The remaining nonparametric methods include the feature-based Vonoroi, PointNet++, Alpha Complex and Random Flag Complex liftings implemented in Telyatnikov et al. (2024). 4) Graph Input, Euclidean Output: No known parametric unlabeled approaches exist. Node2vec (Grover and Leskovec, 2016), Deepwalk (Perozzi et al., 2014), Diff2Vec (Rozemberczki and Sarkar, 2018) and Tang and Liu (2009) perform node-level graph embeddings. Graph-level embedding models include invariant embedding (Galland and Lelarge, 2019). Role2Vec embeds the vertex-types (i.e., structural "roles") that multiple vertices are first mapped into (Ahmed et al., 2022).
Parametric labeled graph models include the Graph Autoencoder and graph Variational autoencoder Kipf and Welling (2016), which can be made Bayesian by using the Full Graph Variational autoencoder. These models implement graph node and feature embedding. D-VAE introduces an graph embedding autoencoder for directed acyclic graphs with node features (Zhang et al., 2019). Probabilistic Relational PCA performs graph node embedding using a probabilistic framework (Li et al., 2009). In terms of nonparametric models, Role2Vec, Graph2Vec and invariant embedding can accomodate graphs with or features. Principal component analysis of a graph extends PCA to weighted graph structures, embedding graph nodes (Saerens et al., 2004).
GraRep embeds graph nodes for graph with edge weights and no node features (Cao et al., 2015). For full graph embedding, (Wang et al., 2021) implement a diffusion-wavelet characterization of node-feature distributions for graphs with node features only.
Parametric methods for embedding entire simplicial or cell complexes, with or without features, have been developed through simplicial and cellular autoencoders (Hajij et al., 2020a(Hajij et al., , 2022)). When features are absent, structural attributes of the complex can be used to define node features. Among non-parametric approaches, extensions of node2vec to the simplicial domain have been proposed, including simplex2vec (Billings et al., 2019) and ksimplex2vec (Hacker, 2020), both relying on random walks, an idea also explored in (Schaub et al., 2020). Neural k-Forms (Maggs et al., 2024) represent simplicial complexes as Euclidean embeddings by integrating learnable differential forms over the embedded ksimplices. Thus, this method builds vector representations through geometric integration.
6) Hypergraph Input, Euclidean Output: Among methods embedding hypergraphs with or without features, we find the heterogeneous hypergraph VAE (parametric) (Fan et al., 2022), with a nonparametric alternative in (Zhou et al., 2006). Other parametric approaches such as (Hong et al., 2016;Kajino, 2019) require features on the hypergraphs' nodes, while other nonparametric approaches exclusively embed hypergraph structures (Pinder et al., 2021;Turnbull et al., 2023;Gong et al., 2023).
This concludes our review and categorization of non-Euclidean machine learning methods in regression and latent embeddings. For a more detailed review of topology in machine learning, we refer the reader to (Hensel et al., 2021). While several deep neural network models-such as variational autoencoders-appear in our latent embedding taxonomy, we have saved a more complete treatment of deep learning for the next section. Deep neural networks stand out from more traditional machine learning methods in that they are flexible compositions of functions that progressively transform data between spaces. In our consideration of latent embedding methods, we abstracted away from the transformations performed by individual neural network layers to consider only the structure of the input and latent spaces (which may be many layers deep in a model). In the next section, we explicitly consider the input-output structure of individual neural network layers, and the ways in which topology, geometry, and algebra have been incorporated into these layers.
V. SURVEY: NON-EUCLIDEAN DEEP LEARNING
We now review non-Euclidean structures in deep learning, particularly focusing on how geometry, topology, and algebra can enrich the structure of a given layer within a deep neural network. A neural network layer is a function f : X → Y , and thus can be analyzed and categorized in terms of the mathematical structure of the input space X and output space Y . We first cover neural network layers without an attention mechanism (see the first paragraph in Section V-B for an introductory description of attention), followed by layers with attention. We note that we will focus on the explicit mathematical structure of the input and output spaces of each neural network layer, rather than the emergent properties of their latent representations -leaving the analysis of the latter to future work.
Figures 11, 12, and 13 organize deep learning methods into a taxonomy based on the mathematical properties of the input and output of neural network layers (first two columns), as well as on the properties of the layer model (third column). The rows show different types of layers that have been published in the literature or in preprints.
In this section, we categorize neural network layers that consider their inputs and outputs as coordinates in spaces equipped with geometric structures. The first category, layers with Euclidean input and Euclidean output, in row 1 of Fig. 11 is exemplified by the Perceptron layer (Rosenblatt, 1958). This foundational layer is commonly used as a component in a deep neural network comprised of several identical layers: the celebrated Multi-Layer Perceptron (MLP).
Next, we consider layers with Euclidean input with manifold output (row 2 of Fig. 11). The Perceptron-Exp layer Miolane and Holmes (2020) extends the Perceptron layer to produce outputs on manifolds. Here, Exp denotes the Riemannian exponential map, which is applied to the result of the Perceptron. Indeed, Exp is an operation that maps tangent vectors to points on a manifold, i.e., that maps an input in an Euclidean space to an output on the manifold. Only the Perceptron component of this layer has learnable weights; the manifold needs to be known a priori in order to specify and implement the appropriate exponential map. Additionally, in general, there is no analytical expression for the Exp map, which needs to be computed numerically. To avoid this computational cost, this layer is best implemented for manifolds whose Exp enjoys an analytical expression. The third category, manifold input with Euclidean output, is represented by the Log-Perceptron layer Davidson et al. (2018) (row 3 of Fig. 11). This layer generalizes the Perceptron to accept inputs from manifolds, applying the Riemannian logarithm map (Log) prior to the Perceptron. The Log operation converts points on manifolds into tangent vectors, effectively serving as the inverse of the Exp map. Thus, it can be viewed as the inverse of the Perceptron-Exp layer Miolane and Holmes (2020). Here again, knowledge of the manifold is essential for implementation, and preference is given to manifolds with an analytically expressible Log.
Finally, we consider the manifold input with manifold output configuration (row 4 of Fig. 11). This is first illustrated by the Bimap and SPDNet layers from Huang and Gool (2017), which focus on symmetric positive definite (SPD) matrices constrained to the manifold of SPD matrices. More generally, ManifoldNet (Chakraborty et al., 2020) builds on the reformulation of convolutions as weighted Fréchet means of Pennec et al. (2006) to propose a layer whose inputs and outputs are both coordinates on a manifold. In this layer, a weighted mean of the inputs is computed, where the weights are learned with classical backpropagation. We note that, in general, there is no analytical expression for the weighted Fréchet mean, which needs to be obtained by optimization. To avoid this computational cost, approximations using tangent means are also considered. for a detailed study of this field, and to Kondor and Trivedi (2018) for foundational theoretical results on equivariance with respect to any compact group. This line of work builds the symmetries natural to a data domain, such as rotation symmetries for image classification, into the structure of the model. This facilitates weight sharing across different transformations so that the same convolutional filter can be used to detect a given feature in an image in, for example, all orientations. To accomplish this, the input and output spaces of these layers are equipped with group actions, and the layers are defined to be compatible with these actions. Intuitively, if a layer is equivariant to a group action, this means that, if the layer produces an output y for a given input x, then it should also produce a transformed y for any transformed x-where transformed here means "acted upon by the same group element."foot_1 2023) illustrate this configuration. The work by (Finzi et al., 2021) allows us to build an Equivariant-Perceptron layer given any matrix group action on inputs and outputs. Next, we consider the same layer setup except with data as signals in space, instead of coordinates (row 2). In this case, the input and output could be images or feature maps, defined over the domains R n and R m , respectively. The classic convolutional layer (LeCun et al., 1998) is a layer with translation equivariance: a translation of the input image x yields a translated feature map y as output. Regular steerable convolutional layers (Cohen and Welling, 2017) generalize this approach beyond the group of translations. Here, the adjective "regular" means the group action is only on the domain of the input and output signals. By contrast, row 3 shows layers for which both the domain and codomain of each signal are equipped with a group action, such as the Steerable Convolutional layer (Cohen and Welling, 2017).
A different class of layers considers signals defined over Lie groups G p , where we recall that a Lie group is a manifold that is also a group. Specifically, the first layer of the Group Convolutional Network Cohen and Welling (2016) (row 4) takes as input a Euclidean signal, typically an image, and outputs a signal equipped with group action, y :
The second layer of this network (row 5) takes this signal as an input and outputs a new signal equipped with group action.
The group action here is the group composition.
Next, layers in rows 6 and 7 consider signals of the form x, y : G p /H → R n . Here, the domain G p /H defines a socalled homogeneous manifold. This is a manifold equipped with a group action such that, for every pair of points on the manifold, there exists a group element that can transform one point onto the other via the action. For example, the twosphere S 2 is homogeneous for the group of 3D rotations, leading to the introduction of spherical CNNs (Cohen et al., 2018;Kondor et al., 2018). In Fig. 12 row 6, General Steerable Convolutional Layer with the so-called quotient representation (Cohen and Welling, 2017) falls in this category, equipping the domain with group action. In Fig. 12 row 7, (Cohen et al., 2019b) further equips the Euclidean codomains of both input and output signals.
The last category of layers (Fig. 12 rows 8, 9) considers input and output signals with manifold domains and Euclidean codomains. Signals are thus represented as x, y : M → R n .
Here, the concept of gauge equivariance replaces the group equivariance. Gauge equivariance describes the idea of being agnostic to the orientation of a local coordinate system on the manifold of interest M . Cohen et al. (2019a) proposes such a channel-wise gauge convolutional layer. Specifically, the layer is defined such that for a given input x and output y, a different choice of local coordinate system on the input signal's manifold, i.e. its gauge, will yield an equally transformed output. Unlike the global transformation imposed by a group action, such as rotation, these layers deal with preserving local transformations. These layers can be generalized to consider group actions on the codomains of the signals, as does the general gauge convolutional layer with induced representation (Cohen et al., 2019a).
Topology in Neural Network Layers: Here, we categorize neural network layers that consider their inputs and outputs as signals over a topological domain. We refer the reader to the survey by Papillon et al. (2023) for a comprehensive overview of the neural network layers within this category, and we focus only on illustrative examples. These layers are all equivariant to the permutation of the elements of their topological domains: permutation of the points in a set, of the nodes in a graph, etc. As such, all these layers are, at a minimum, equipped with group action on their domain.
We begin with layers that consider Euclidean signal on a set, depicted in the first row of Fig. 13. PointNet (Qi et al., 2017a) processes point clouds, which have signal of the form x, y : P → R n where P is a set. The model is equivariant to a permutation in P through its use of a symmetric function (max pooling). PointNet++ (Qi et al., 2017b) operates on the same signal, using a series of PointNet layers to learn patterns at different scales by imbuing hierarchy within the point cloud. In both of these works, the experiments are restricted to run only on point clouds in 2D or 3D. Deep Sets (Zaheer et al., 2017) characterizes what is necessary and sufficient in terms of parameter sharing for a layer to be equivariant (two examples are summation and max pooling).
We can extend these layers to also equip the signal's codomain with group action (row 2), instead of just the domain. Tensor Field Networks (TFN) (Thomas et al., 2018), and PONITA (Bekkers et al., 2024) process signals in this category. Like PointNet++, TFN and PONITA process 2D and 3D point clouds and are equivariant to the permutation of their points in P . TFN and PONITA are additionally equivariant to 3D rotations and translations.
Going beyond a simple set, the next set of layers considers signals defined over graphs (Fig. 13 rows 3, 4). These layers process signals of the form x, y : G → R n , i.e., defined over a graph G. This category is exemplified by the Graph Convolutional layer (Kipf and Welling, 2017) and Message Passing layers (Gilmer et al., 2017). Such layers can be generalized to also consider a group action on the codomain of the signals (row 4). We find, for example, the E(n-Equivariant Graph Neural Networks (EGNN) by Satorras et al. (2021a).
What if the underlying topology of the data is more accurately represented by multi-way relationships, rather than the pairwise connections of graphs? The following works address this by leveraging richer topological spaces, denoted as Ω, as their signal's domain. Rows 5 and 6 show layers that process signals defined over simplicial and cellular complexes, which both allow for hierarchical relationships, i.e. faces, between edges.
Examples include simplicial convolutional (Ebli et al., 2020), cellular convolutional (Roddenberry et al., 2022), simplicial message passing (Bodnar et al., 2021), and cellular message passing (Hajij et al., 2020b) layers. In the case where group action also equips the co-domain (row 6), we find, for example, the E(n)-Equivariant Message Passing Simplicial Networks (EMPSN) (Eijkelboom et al., 2023) and the Clifford group Equivariant Message Passing Simplicial Networks (EMPSN) (Liu et al., 2024). These both introduce group actions on the codomain, but for different groups.
In the same spirit, row 7 describes layers that process signals defined over the hypergraph domain, which allows for multi-way edges, i.e., hyperedges, connecting many nodes at once. Examples include the Hypergraph Convolutional layer (Arya et al., 2020) and the Hypergraph Message Passing layer (Heydari and Livi, 2022). The last topological space that arises in the field is the combinatorial complex, which combines the hierarchical relationships of cellular complexes with the set-wise flexibility of hypergraphs (row 8). Examples of neural network layers that process this type of signals are the Combinatorial Convolutions (Hajij et al., 2023) and Combinatorial Message Passing (Hajij et al., 2023). We can further leverage algebraic structure by equipping the codomain with a group action (row 9). The E(n)-equivariant topological neural networks from Battiloro et al. (2025) are an example of this configuration. The layers of this network process geometric features in the codomain R n , such as velocities or positions associated with elements of Ω.
We now review mathematical structures in neural network layers that leverage the attention mechanism, focusing on how non-Euclidean structures can enrich the structure of the attention coefficients and the attention layers. Attention mechanisms emerged as a transformative approach with foundational works by Graves et al. (2014Graves et al. ( , 2016) ) introducing dotproduct attention, and Bahdanau et al. ( 2015) applying it to machine translation. It was later widely popularized by the Transformer architecture (Vaswani et al., 2017). At its core, attention addresses the fundamental challenge of selectively focusing on relevant parts of input data while creating direct connections between arbitrary positions, thereby overcoming the limitations of traditional architectures in handling longrange dependencies. We devote a section in our study to attention mechanisms because they represent a distinct and increasingly dominant paradigm in neural network design, with unique mathematical properties that complement traditional approaches in our taxonomy.
Let us briefly unpack how attention works. In a layer, the attention coefficient α is computed from a query q and a key k. Hence, we examine the structure of the key k and the query q inputs, represented as signals over a domain. For example, the traditional transformer (Vaswani et al., 2017), depicted in Fig. 14, considers keys and queries as functions over a one-dimensional domain R representing time. The output attention coefficient α is represented as a signal over the product domain: in the transformer example, it is a function the product domain R × R. Second, the attention layer transforms input values v to output values v ′ through the attention coefficients α. Hence, we look at the mathematical properties of v and v ′ , both represented as signals over a domain. In the classical transformers, they are signals over the time domain R.
Another example of a classical transformer is the Vision Transformer (Dosovitskiy et al., 2021), which divides an image into a sequence of image patches: hence, patches in R n over R. We highlight the difference between the mathematical representation of these signals, and their computational representation during an actual implementation of the transformer. Mathematically, we represent the signals' domain as the continuous real line R. Computationally, however, this real line is discretized into T steps and associated T discrete tokens (for either words or image patches). Yet, the representation as the real line is useful to unify the original transformer with the more complicated layers introduced next.
With the classical case in mind, we now turn to Figures 15, 16 The key k and the query q are the inputs to the attention coefficient α; the value v is the input to the attention layer, and the output value v ′ is the weighted result of that layer. Inputs and outputs are represented as signals, i.e., as functions from a domain to a codomain. Notations: R n : Euclidean space; M : Manifold. into a taxonomy based on the mathematical properties of their non-Euclidean inputs and outputs. Each row highlights the properties of the domains and codomains of the key k, query q, coefficient α, input value v and output value v ′ signals.
We consider the case where keys and queries are defined as manifold signals on a Euclidean domain (Fig. 15 row 1). This row introduces geometry in the codomains of the keys and query signals, which write k, q : R → M . Meanwhile, the attention coefficients α, the input and output values v, v ′ have the same structure as in the classic transformer. The multimanifold attention mechanisms by Konstantinidis et al. (2023) illustrates this configuration. Specifically, this work considers the classical attention coefficient α as the computation of a Euclidean distance between key and query. Accordingly, their proposed geometric attention coefficient replaces the Euclidean distance by a Riemannian geodesic distance between key and query, which are interpreted as elements of a manifold: the manifold of SPD (symmetric positive definite) matrices, the Grassmann manifold, or both-hence the term "multi"-manifold. We note that the input data to this transformer architecture is still Euclidean, since this attention mechanism is proposed for images in vision transformers. However, the way this data is processed by the transformer's internal layers is non-Euclidean.
In Fig. 15 row 2, we consider the case in which queries, keys and values are first mapped from Euclidean activations Layer Input Equiavariance in Attention Mechanisms Layer Output Layer Operation onto the hyperboloid manifold. The hyperbolic transformer (Gulcehre et al., 2019) computes attention weights via an exponential (or sigmoid) of the hyperbolic distance between q i and k j , and aggregates Klein-model values with the Einstein midpoint. Although raw inputs such as images, graph node features or word embeddings start in Euclidean space, all attention operations in the network occur in hyperbolic space.
Equivariance in Attention Mechanisms: We first consider layers where keys, queries and values are defined as signals on Euclidean domains with group action on the codomain. Fig. 16 row 1 illustrates an attention mechanism that includes additional algebraic structure in both keys, queries, input and output values. The Geometric Algebra Transformer represents this configuration Brehmer et al. (2023). Its layers were designed to process "geometric data" defined as scalars, vectors, lines, planes, objects and their transformations (e.g., rotations) in 3D space. Such geometric data is encoded into multivectors, which are elements of the projective geometric algebra also called the Clifford algebra. For simplicity, we consider this space as a Euclidean space R n with algebraic structure. The attention coefficients are group invariant, while the attention layer is group equivariant, for the Euclidean group E(3) of translations, rotations and reflections in 3D space. This configuration is also illustrated by the Steerable Transformer (Kundu and Kondor, 2024) which processes Euclidean codomains R n with equivariance to the special Euclidean group SE(n), for application to image processing and machine learning tasks.
In Fig. 16 row 2, we generalize row 1 to signals defined over manifold domains. As before, this row also introduces geometric structure in the keys, queries, values. In contrast to the previous row, however, this layer brings geometry into the domains of the signals: k, q, v, v ′ whereas the above layer brought geometry in their codomains. Consequently, for this row, the attention coefficients α : M × M → R have the product manifold M × M as their domains. This configuration is illustrated in the Lie Transformer (Hutchinson et al., 2021), where the manifolds of interest are Lie groups and their subgroups. We note that the data processed by this architecture does not have to belong to a Lie group; only to be acted upon by a Lie group. A lifting layer is introduced to convert the raw data into Lie group elements, which are then handled by the Lie Transformer. The attention layer is then equivariant.
Lastly, we explore attentional layers that feature built-in gauge equivariance and invariance (Fig. 16 row 3), rather than the group equivariance from before. The Gauge invariant transformer in He et al. (2021) presents such a configuration: the attention coefficients are gauge invariant, and the attention layer is gauge equivariant. This work exclusively focuses on two-dimensional manifolds M embedded in 3D Euclidean space.
Topology in Attention Mechanisms: We now turn to attentional layers defined on topological spaces. All of these layers are permutation equivariant, meaning they respect domain-level group actions. Their key differences lie in how they define the underlying topological domain on which the data resides. Following the same approach as in the non-attentional case, we will present these domains in roughly increasing order of topological complexity, starting with the simple set. č ć Fig. 17. Topology in Attention Mechanisms categorized according to the mathematical properties of the attention coefficients and of the attention layer. Notations: R n : Euclidean space; M : Manifold; P : point set; G: graph; Ω: topological space.
between keys and queries: either a graph-based geodesic distance, or a Riemannian geodesic distance on an oblique manifold. The latter manifold refers to the set of matrices whose columns are unit norm but not necessarily orthogonal.
It is a useful choice because the unit norm constraint helps stabilize optimization, preventing issues like exploding or vanishing gradients, while offering more flexibility than manifolds that enforce full orthogonality.
Slightly increasing topological complexity, Fig. 17 rows 3 and 4 describe layers defined on the graph domain. The most wellknown example is the Graph Attention Transformer (GAT) (Veličković et al., 2018), which features the same group action (permutation) equivariance on the domain (row 3). We can additionally equip the codomain of a graph attentional layer with group action (row 4), as is the case in the SE(3)-Transformer (Fuchs et al., 2020). Here, the codomain of the signals is additionally restricted to R 3 , equipped with an action of the group of translations and rotations in 3D SE(3). This transformer was proposed to process 3D point clouds, and provides SE(3)-invariant attention coefficients and SE(3)equivariant attention layer.
Going beyond pairwise relations, the remaining rows of Fig. 17 consider the same richer domains as outlined in Fig. 13 rows 5-9 and detailed in the survey Papillon et al. (2023).
While they all feature group action (permutation) equivariance on the domain, none extend algebraic structure to the codomain, as is the case for graphs (Fig. 17 row 4). We briefly detail the works that exemplify this line of work: Simplicial Graph Attention Network (SGAT) (Lee et al., 2022) and Cell Attention Networks (CAN) (Giusti et al., 2023) define their signals on simplicial and cellular complexes, respectively. The Cellular Transformer (Ballester et al., 2024) additionally includes positional encodings. Dynamic Hypergraph Neural Networks (DHGNN) (Jiang et al., 2019b) and Hypergraph Attention Networks (Hyper GAT) (Ding et al., 2020) leverage hypergrpahs. The Higher Order Attention Network (HOAN) architecture (Hajij et al., 2023) uses the combinatorial complex domain, which combines the relations featured in cellular complexes and hypergraphs.
This concludes the review of non-Euclidean in deep neural network layers. While a great diversity of layers and mechanisms have been proposed, we hope that our illustrated taxonomy aids researchers in understanding the landscape and identifying opportunities for innovation and application. In the next sections, we turn to practical aspects of deploying topological and geometric machine learning methods in applications, including a table of common benchmarks used in the literature, a table of non-euclidean software libraries, and a review of key domains in which these approaches have been applied.
Here, we present a brief review of the benchmarks that have been considered in the non-Euclidean deep learning literature, compiling results from a broad sample of neural networks with topological, geometric, and algebraic layers in Table I, and highlighting the diversity of tasks and datasets used in the literature.
Tasks and Datasets: We first observe that a wide variety of task and benchmark datasets have been used in the literature, with little overlap between models. In other words, it is rare that two different models have been benchmarked on the same dataset. This is not surprising, since different models use different geometric, topological, and algebraic structures and different structures are well suited for different tasks.
There are, however several benchmarks that appear across models: MNIST and CIFAR for image classification, and Cora, Citeseer, and Pubmed for graph classification. Many geometrical models are tested by examining how well they model dynamical or physical systems. These results are not easily comparable across models, as the tasks are often customized each paper.
A key benefit of building mathematical structure into neural networks is that it constrains the hypothesis search space. If the structure is well matched to the problem, the model should require fewer parameters and fewer computations. Many papers mention this, but only a few report the number of parameters (see right column of Table I). As parameter and data efficiency are frequently cited as advantages of building structure into neural network models, we encourage authors to more regularly report parameter counts and computational cost in their papers along with performance metrics.
Table II highlights publicly available software libraries that make the methods of this field computationally accessible. Here, we limit our discussion to libraries whose commit history suggests continued development and have a following indicated by at least 50 Github Stars. As shown by the number of stars and actively developed repositories, packages for topological methods are the most well developed, including important engineering foundations such as CUDA and C++ accelerated network primitives, and large collections of model implementations that continue to be maintained. The library ecosystem for geometric learning methods is quickly growing in interest and contributors, extending the packages beyond optimizers over specific manifolds to more general differential geometry tools. While the packages for algebra in machine learning are the most nascent, there have been exciting new developments within the past few years in making more specialized packages for accelerating group convolutions and other algebraic operations as the need for more specialized applications have emerged. VIII. NON-EUCLIDEAN LEARNING FOR SCIENCE Many problems in science and engineering are intrinsically non-Euclidean and thus provide an exciting opportunity for the application of non-Euclidean ML methods. Here, we briefly highlight key developments in selected application areas. We refer the reader to Wu et al. (2021); Bronstein et al. (2021);
TABLE I APPLICATIONS AND BENCHMARK OF NEURAL NETWORKS WITH GEOMETRIC, TOPOLOGICAL AND ALGEBRAIC STRUCTURES. WE ORGANIZE MODELS ACCORDING TO WHETHER IT USES ATTENTION AND THEIR GEOMETRIC, TOPOLOGICAL AND ALGEBRAIC STRUCTURE, WITH THE ABBREVIATIONS: M: MANIFOLD, Gp: GROUP, S: SET, G: GRAPH, Ω: TOPOLOGICAL DOMAIN, A: ALGEBRA. MODELS ARE ALSO ORGANIZED BASED ON WHICH TASK THEY PERFORM, AND ON WHICH BENCHMARK DATASETS. WE INCLUDE ACCURACIES FOR BENCHMARKS THAT TWO OR MORE MODELS USE, CONVERTING TEST ERROR TO ACCURACY WHEN NEEDED, ALONG WITH STANDARD ERROR IF REPORTED. MODEL PARAMETERS ARE LISTED IF THE PAPER REPORTS THEM. N. R. MEANS NOT REPORTED. Model Structure Task Benchmark datasets # Params Without Attention Riemannian VAE Miolane19 M Dimension Reduction Human Connectome Project (HCP) N.R. S-VAE/VGAE Davidson18 M Latent representation for image classification and link prediction MNIST (93.4± 0.2*), Cora (94.1±0.3), Citeseer (95.2±0.2), Pubmed (96.0±0.1) N.R. SPDNet Huang,VanGool16 M Visual classification (emotion, action, face) AFEW, HDM05 and PaSC N.R. EMLP Finzi21 Gp Dynamical modeling Double pendulum N.R. LeNet-5 LeCun98 Gp Image classification MNIST (99.2±0.1) N.R. Steerable CNN Cohen,Welling17 Gp Image classification CIFAR (10: 76.3; 10+: 96.4; 100+: 81.2) 4.4M 9.1M G-CNN Cohen,Welling16 Gp Image classification Rotated MNIST, CIFAR (10: 93.5; 10+: 95.1) 2.6M G-CNN Cohen19a Gp Climate, pointcloud segmentation Climate Segmentation, Stanford 2D-3D-S N.R. E(n)-EGNN Satorras2021 Gp Molecular property prediction, dynamical modeling QM9, N-body, Graph autoencoder N.R. PONITA Bekkers2024 Gp Molecular property prediction and generation, dynamical modeling rMD17, QM9, N-body N.R. PointNet++ Qi17 S Image, 3D, scene classification MNIST (99.49), ModelNet40 (91.9), SHREC15, ScanNet 1.7M Tensor field network Thomas18 S 3D-point-cloud prediction QM9 N.R. GCN Kipf,Welling17 G Link prediction Cora, Citeseer, Pubmed, NELL N.R. enn-s2s Gilmer17 G Molecular property prediction QM9 N.R. SNN Ebli20 Ω Coauthorship prediction Semantic Scholar Open Research Corpus N.R. MPSN Bodnar21 Ω Trajectory, graph classification TUDataset N.R. CXN Hajij20 Ω --N.R. HMPNN Heydari22 Ω Citation node classification Cora (92.2) N.R. CCNN Hajij23 Ω Image segmentation, image, mesh, graph classification Human Body, COSEG, SHREC11 N.R. E(n)-EMPSN Eijkelboom2023 Ω Molecular property prediction, dynamical modeling QM9, N-body 200K Clifford-EMPSN Liu2024 Ω Pose estimation, dynamical modeling CMU MoCap, MD17 200K E(n) Equivariant TNN Battiloro2024 Ω Molecular property, air pollution prediction QM9, Air Pollution Downscaling 1.5M With Attention Transformer Vaswani17 -Machine translation WMT 2014 N.R. MMA ViT Konstantinidis23 M Image classification, segmentation CIFAR (10: 94.7, 100+: 77.5), T-ImageNet, ImageNet, ADE20K 3.9M GATr Brehmer23 A Dynamical modeling N-body, artery stress, diffusion robotics 4.0M Steerable Transformer Kundu2024 Gp Point-cloud, Image classification Rotated MNIST (99.03), ModelNet10 (90.4) 0.9M Lie Transformer Hutchinson20 Gp Regression, dynamics QM9, ODE spring simulation 0.9M GET He21 M Shape classification, segmentation SHREC07, Human Body Segmentation 0.15M Set Transformer Lee19 S Max value regression, clustering Omniglot, CIFAR (100: 0.92±0.01 N.R. PCT Guo21 S Point-cloud classification, regression, segmentation ModelNet40 (93.2), ShapeNet (86.4), S3DIS 1.4M GSA Li22 S Object classification, segmentation ModelNet40 (93.3), ScanObjectNN, ShapeNet (85.9) 18.5M SE(3)-Transformer Fuchs20 G Dynamics, classification, regression N-body, ScanObjectNN, QM9 N.R. GAT Veličković18 G Link prediction Cora, Citeseer, Pubmed, PPI N.R. CAN Giusti23 Ω Graph classification TUDataset N.R. Cellular Transformer Ballester24 Ω Graph classifical, Graph regression GCB, Zinc, Ogbg Molhiv N.R. SGAT Lee22 Ω Node classification DBLP 2 , ACM, IMDB N.R. DHGNN Jiang19 Ω Link, sentiment prediction Cora (82.5), Microblog 0.13M HyperGAT Ding20 Ω Text classification 20NG, R8, R52, Ohsumed, MR N.R. Geometry Packages Domains Core Features Stars GeomStats 2020 Manifolds, Lie Groups, Fiber Bundles, Shape Spaces, Information Manifolds, Graphs Manifold operations, Algorithms, Statistics, Optimizers 1.3k GeoOpt 2020 Manifolds Layers, Manifold operations, Stochastic optimizers for deep learning 917 PyManOpt 2016 Manifolds, Lie Groups Manifold operations, Optimizers 816 GeometricKernels 2024 Riemannian manifolds, graphs and meshes Gaussian process models 247 PyRiemann 2023 SPD Matrices Machine Learning, Data Analysis for biosignals 676 Topology Packages Domains Core Features Stars PytorchGeometric 2019 Graphs Baseline Models, Layers, Fast Basic Graph Operations, Datasets, Dataloaders 22.3k NetworkX 2008 Graphs, Digraphs, Multigraphs Data structures, Graph generators, Graph Algorithms, Network Analysis Measures 15.7k DGL 2019 Graphs Baseline Models, Layers, Fast Basic Graph Operations, Datasets, Dataloaders, Framework-agnostic (PyTorch, Tensorflow, etc. are swappable) 13.9k DIG 2021 Graphs Baseline models, Datasets, Evaluation Metrics 1.9k AutoGL 2021 Graphs Neural Architecture Search, Hyper-Parameter Tuning, Ensembles 1.1k HyperNetX 2024 Hypergraphs Machine Learning Algorithms, Analysis, Visualization 606 DHG 2022 Graphs, hypergraphs, bipartite graphs, hypergraphs, directed hypergraphs, ... Models, Basic Operations, Dataloaders, Visualization, Auto ML, Metrics, Graph generators 737 TopoModelX 2024 Graphs, colored hypergraphs, complexes Baseline Models, Layers, Higher-order message passing 205 TopoNetX 2024 Graphs, colored hypergraphs, complexes Topography Generators, Computing topological properties, Arbitrary cell attributes 268 TopoEmbedX 2024 Graphs, colored hypergraphs, complexes Representation learning, embeddings 80 TopoBench 2024 Graphs, hypergraphs, complexes Benchmarks, lifting, dataloaders, losses, training framework 116 XGI 2023 Hypergraphs, directed hypergraphs, symplical complexes Graph generators, metrics, algorithms, dataloaders, visualization 209 Algebra Packages Domains Core Features Stars 2022 E(3) Equivariant Feature Fields Group Convolutions, Steerable Group Convolutions 1.1k ESCNN & E2CNN 2022 E(n) Equivariant Feature Fields, Graphs Group Convolutions, Steerable Group Convolutions 584 3 NequIP 2022 E(3) Equivariance on Graphs Group Convolutions, Steerable Group Convolutions 712 EMLP 2021 Matrix Groups, Tensors, Irreducible Representations, Induced Representations Programmatic generation of equivariant MLPs for arbitrary matrix groups in JAX 267 PyQuaternion Quaternions Quaternion operations, rotation representation conversions, differentiation, integration 357 TABLE II SOFTWARE PACKAGES FOR MACHINE LEARNING WITH TOPOLOGY, GEOMETRY, AND ALGEBRA. WE ORGANIZE PACKAGES ACCORDING TO THE MATHEMATICAL, NON-EUCLIDEAN STRUCTURES THEY FOCUS ON.
Graph neural networks have become a workhorse for molecular analysis, treating molecules as graphs with atoms as nodes and bonds as edges (Gilmer et al., 2017;Bronstein et al., 2021) (Card C3 in Figure 4). Progress in this field has largely involved the construction of message-passing neural networks with favorable properties, such as equivariance to a growing family of group transformations (see examples of layers in Fig. 13), novel forms of weight sharing, more expressive primitives, and more efficient formulations for parameterization and computation (Schütt et al., 2017;Thomas et al., 2018;Batzner et al., 2022;Satorras et al., 2021b;Bekkers et al., 2024). Deep networks with geometric structure have also been used directly for drug screening to discover new antibiotics (Stokes et al., 2020).
Recently, deep equivariant generative modeling has emerged as a powerful framework for molecule synthesis. Prior work by Gebauer et al. (2019); Simonovsky and Komodakis (2018); Simm et al. (2021) establishes the importance of leveraging geometric properties for the synthesis of molecules. Hoogeboom et al. (2022) introduces equivariant denoising diffusion models for molecule generation by directly generating 3D atomic coordinates, demonstrating improved quality and efficiency. This was recently extended by Xu et al. (2023) to perform equivariant diffusion over a molecular latent space, and by Vignac al. (2023) which achieves much higher stability for generated molecules on the GEOM-DRUGS dataset. Another line of work generates molecular invariants such as angles and distances, which are then used to produce coordinates (Luo and Ji, 2022). Recent work has also demonstrated the importance of equivariances and invariances for molecular conformer generation (Xu et al., 2022;Reidenbach and Krishnapriyan, 2025).
Computer vision entails the inference of properties of the visual world from images or other measurements such as LIDAR. There are many subtasks in computer vision, such as object recognition, semantic segmentation, image and video generation, and depth estimation. Historically, network primitives that capture the topological structure and symmetries of images have dominated vision benchmarks, including CNNs, GCNNs, and Vision Transformers (ViTs) (LeCun et al., 1998;Krizhevsky et al., 2012;Cohen and Welling, 2016;Dosovitskiy et al., 2021). Within our taxonomy, CNNs operate on regular image grids and process Euclidean signals on Euclidean domains (Card S1, Figure 5). When translation equivariance is built in-as in standard CNNs-or extended to include rotational symmetries via group convolutions (e.g., GCNNs), these models align with Cards S7 and S10. ViTs similarly operate on Euclidean domains (Card S1) but with a different architectural inductive bias, typically treating images as sequences of patches with learned positional encodings. When symmetry-preserving mechanisms such as equivariant attention are introduced (see Figure 16), ViTs may extend toward Card S10, provided both the positional structure and patch embeddings transform under a group action.
Another successful application of non-Euclidean deep learning in computer vision has been graph neural networks. Specifically, these are implemented on data native to or lifted to point-clouds. Pioneering works such as Pointnet++ and PointTransformer introduced graph-structured deep networks as breakthrough methods in 3D semantic segmentation and object detection at whole-room scales (Qi et al., 2017b;Engel et al., 2020). Another promising computational primitive, Slot Attention, introduces a novel messaging-passing strategy to perform unsupervised object discovery using permutation invariant slots, which incorporates spatial symmetries using slot-centric reference frames (Locatello et al., 2020;Biza et al., 2023).
Biomedical imaging involves inferring the structure of biological tissues from measurements of their physical properties, typically in the form of electromagnetic fields, acoustic waves, and other physical phenomena. Quantities of interest include shape, composition, or internal state. Thus, geometric and topological structure play an important role in their analysis.
Many machine learning problems in medical imaging require reasoning about 3D structures, including their shape, their variations throughout a population, and changes throughout time. As tissue states are non-Euclidean, their statistics and evolution require geometric treatment (Pennec et al., 2019). Variations in organ shapes lie on low-dimensional manifolds, and geometry-aware dimensionality-reduction methods such as tangent PCA (Boisvert et al., 2008) or Principle Geodesic Analysis (PGA) enable meaningful representations for downstream tasks (Fletcher et al., 2004;Fletcher and Joshi, 2007;Hinkle et al., 2012a). We refer to row 3 of Fig. 8 for details on these techniques. Geometric methods have also been applied the analysis of the effects of aging in the corpus collosum with MRI scans (Hinkle et al., 2012b), to brain connectomics data in Diffusion Tensor Imaging (Pennec et al., 2006), and to the segmentation of 3D anatomical structures from CT and MRI scans in the lateral cerebral ventricle, the kidney parenchyma and pelvis, and the hippocampus (Pizer et al., 2003).
Physics data naturally has many symmetries and often takes the form of relations between unordered sets, an ideal setting for topological and equivariant methods. Dynamics between particles or nodes in a mesh can be effectively computed using learned graph message passing for various types of physics data (Sanchez-Gonzalez et al., 2020;Pfaff et al., 2021). Equivariant transformer (Fig. 16, row 1) and graph neural network architectures (Fig. 13, row 4) have been successfully applied to data analysis for the Large Hadron Collider and other simulations such as gravity for the n-body problem (Fuchs et al., 2020;Brandstetter et al., 2022). Topological methods are well suited to process the hundreds of petabytes of highly relational data produced by experiments from the Large Hadron Collider and have demonstrated their utility for the next stage of fundamental discoveries in particle physics (DeZoort et al., 2023). Recent work (Brehmer et al., 2023) proposes an equivariant transformer architecture processing embedded geometric representations, and demonstrates its efficacy on mesh interaction estimation and n-body simulations. Astrophysics data is also well suited for application of equivariant networks. Some examples include the classification of radio galaxies using group-equivariant CNNs (Scaife and Porter, 2021), optimal cosmological analysis using equivariant normalizing flows (Dai and Seljak, 2022), and cosmic microwave background radiation analysis using spherical equivariant CNNs (McEwen et al., 2022).
Looking ahead, many scientific challenges, including those reviewed above, increasingly require not only understanding the potentially non-Euclidean structure of data, but also modeling how that structure evolves over time. From the folding and interaction of proteins, to the progression of disease in biomedical imaging, to predicting object trajectories and physical interactions in computer vision, the temporal evolution of complex systems is a central concern. While modeling static structure has led to significant improvements in their respective fields, as we reviewed above, incorporating geometry-or topology-constrained dynamics into machine learning is a natural and necessary next step.
Work in theoretical machine learning and dynamical systems theory, specifically on latent variable models, provides a promising foundation for addressing such problems (Fig. 18). Prior studies have shown that across diverse domains the observed variables often live in high-dimensional state spaces with geometric constraints (Fig. 18, left). However, latent variables, whether in natural language processing (Mikolov et al., 2013), neuroscience (Schneider et al., 2023;Dinc et al., 2025), or interpretability research (Shai et al., 2024;Hyvärinen et al., 2024), can often be extracted with dimensional bottlenecks and consequently reside on low-dimensional manifolds, both Euclidean and non-Euclidean (Fig. 18, right). These latent spaces yield compact, interpretable, and physically meaningful representations of system dynamics.
A key opportunity for future work lies in modeling not just these latent states, but their evolution (Fig. 18, bottom). Many current models assume linear (Euclidean) dynamics (Abbaspourazad et al., 2024), but manifold-valued dynamics can capture richer constraints like symmetry, curvature, and conservation laws. As in static settings, incorporating such geometric inductive biases may be crucial for generalization under limited data. Non-Euclidean dynamical systems thus offer a compelling direction for future models grounded in the structures introduced here.
As the availability of richly structured, non-Euclidean data grows, a new paradigm of machine learning has emerged, leveraging the mathematics of geometry, topology, and algebra to extract novel insights. In this review, we have provided an accessible overview of this field, unifying disparate threads in the literature into a common framework. Our illustrated taxonomy contextualizes, classifies, and differentiates existing approaches and illuminates gaps that present opportunities for innovation. In addition, we provide resources for the practitioner's use. Section VI organizes the benchmarks used in the non-Euclidean deep learning literature. Section VII provides a list of core open-source software libraries for non-Euclidean machine learning. Section VIII summarizes the core domains that non-Euclidean machine learning has been applied to thus far. We hope this serves as an invitation for both theoreticians and practitioners to further explore the potential for geometry, topology, and algebra to reshape modern machine learning, just as they reshaped our fundamental understanding of space over a century ago.
We recognize that "algebra" can have a more general meaning in other contexts, such as in categorical deep learning, where it refers to a structure map relative to an endofunctor within a category. However, within this review, we restrict its meaning to the definition provided above.
We note that several authors have proposed categorical deep learning, a broader framework that generalizes equivariant deep learning(Gavranović et al., 2024). However, due to the infancy of this field, a detailed discussion lies beyond the scope of this review.