More Like this

1. Consistent Interpolating Ensembles via the Manifold-Hilbert Kernel

   Wang, Yutong; Scott, Clayton D. (January 2022, Neural Information Processing Systems 2022)

   Recent research in the theory of overparametrized learning has sought to establish generalization guarantees in the interpolating regime. Such results have been established for a few common classes of methods, but so far not for ensemble methods. We devise an ensemble classification method that simultaneously interpolates the training data, and is consistent for a broad class of data distributions. To this end, we define the manifold-Hilbert kernel for data distributed on a Riemannian manifold. We prove that kernel smoothing regression and classification using the manifold-Hilbert kernel are weakly consistent in the setting of Devroye et al. [22]. For the sphere, we show that the manifold-Hilbert kernel can be realized as a weighted random partition kernel, which arises as an infinite ensemble of partition-based classifiers. 

   more » « less
2. Intrinsic Grassmann Averages for Online Linear, Robust and Nonlinear Subspace Learning

   https://doi.org/10.1109/TPAMI.2020.2992392  

   Chakraborty, Rudrasis; Yang, Liu; Hauberg, Soren; Vemuri, Baba (May 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence)

   Principal Component Analysis (PCA) and Kernel Principal Component Analysis (KPCA) are fundamental methods in machine learning for dimensionality reduction. The former is a technique for finding this approximation in finite dimensions and the latter is often in an infinite dimensional Reproducing Kernel Hilbert-space (RKHS). In this paper, we present a geometric framework for computing the principal linear subspaces in both situations as well as for the robust PCA case, that amounts to computing the intrinsic average on the space of all subspaces: the Grassmann manifold. Points on this manifold are defined as the subspaces spanned by K -tuples of observations. The intrinsic Grassmann average of these subspaces are shown to coincide with the principal components of the observations when they are drawn from a Gaussian distribution. We show similar results in the RKHS case and provide an efficient algorithm for computing the projection onto the this average subspace. The result is a method akin to KPCA which is substantially faster. Further, we present a novel online version of the KPCA using our geometric framework. Competitive performance of all our algorithms are demonstrated on a variety of real and synthetic data sets. 

   more » « less
3. Kernel Multiaccuracy

   https://doi.org/10.4230/LIPIcs.FORC.2025.7  

   Long, Carol Xuan; Alghamdi, Wael; Glynn, Alexander; Wu, Yixuan; Calmon, Flavio P (January 2025, Schloss Dagstuhl – Leibniz-Zentrum für Informatik)

   Bun, Mark (Ed.)

   Predefined demographic groups often overlook the subpopulations most impacted by model errors, leading to a growing emphasis on data-driven methods that pinpoint where models underperform. The emerging field of multi-group fairness addresses this by ensuring models perform well across a wide range of group-defining functions, rather than relying on fixed demographic categories. We demonstrate that recently introduced notions of multi-group fairness can be equivalently formulated as integral probability metrics (IPM). IPMs are the common information-theoretic tool that underlie definitions such as multiaccuracy, multicalibration, and outcome indistinguishably. For multiaccuracy, this connection leads to a simple, yet powerful procedure for achieving multiaccuracy with respect to an infinite-dimensional class of functions defined by a reproducing kernel Hilbert space (RKHS): first perform a kernel regression of a model’s errors, then subtract the resulting function from a model’s predictions. We combine these results to develop a post-processing method that improves multiaccuracy with respect to bounded-norm functions in an RKHS, enjoys provable performance guarantees, and, in binary classification benchmarks, achieves favorable multiaccuracy relative to competing methods. 

   more » « less
4. Thermodynamic ensembles with cosmological horizons

   https://doi.org/10.1007/JHEP07(2022)042  

   Banihashemi, Batoul; Jacobson, Ted (July 2022, Journal of High Energy Physics)

   A bstract The entropy of a de Sitter horizon was derived long ago by Gibbons and Hawking via a gravitational partition function. Since there is no boundary at which to define the temperature or energy of the ensemble, the statistical foundation of their approach has remained obscure. To place the statistical ensemble on a firm footing we introduce an artificial “York boundary”, with either canonical or microcanonical boundary conditions, as has been done previously for black hole ensembles. The partition function and the density of states are expressed as integrals over paths in the constrained, spherically reduced phase space of pure 3+1 dimensional gravity with a positive cosmological constant. Issues related to the domain and contour of integration are analyzed, and the adopted choices for those are justified as far as possible. The canonical ensemble includes a patch of spacetime without horizon, as well as configurations containing a black hole or a cosmological horizon. We study thermodynamic phases and (in)stability, and discuss an evolving reservoir model that can stabilize the cosmological horizon in the canonical ensemble. Finally, we explain how the Gibbons-Hawking partition function on the 4-sphere can be derived as a limit of well-defined thermodynamic ensembles and, from this viewpoint, why it computes the dimension of the Hilbert space of states within a cosmological horizon. 

   more » « less
5. Learning Approximate Forward Reachable Sets Using Separating Kernels

   Thorpe, Adam; Ortiz, Kendric; and Oishi, Meeko M. (January 2021, Proceedings of Machine Learning Research)

   null (Ed.)

   We present a data-driven method for computing approximate forward reachable sets using separating kernels in a reproducing kernel Hilbert space. We frame the problem as a support estimation problem, and learn a classifier of the support as an element in a reproducing kernel Hilbert space using a data-driven approach. Kernel methods provide a computationally efficient representation for the classifier that is the solution to a regularized least squares problem. The solution converges almost surely as the sample size increases, and admits known finite sample bounds. This approach is applicable to stochastic systems with arbitrary disturbances and neural network verification problems by treating the network as a dynamical system, or by considering neural network controllers as part of a closed-loop system. We present our technique on several examples, including a spacecraft rendezvous and docking problem, and two nonlinear system benchmarks with neural network controllers. 

   more » « less