More Like this

1. Probabilistic Iterative Methods for Linear Systems

   Cockayne, Jon; Ipsen, Ilse C.F.; Oates, Chris J.; Reid, Tim W. (January 2021, Journal of machine learning research)

   This paper presents a probabilistic perspective on iterative methods for approximating the solution x in R^d of a nonsingular linear system Ax = b. Classically, an iterative method produces a sequence x_m of approximations that converge to x in R^d. Our approach, instead, lifts a standard iterative method to act on the set of probability distributions, P(Rd), outputting a sequence of probability distributions  mu_m in P(Rd). The output of a probabilistic iterative method can provide both a “best guess” for x, for example by taking the mean of  mu_m, and also probabilistic uncertainty quantification for the value of x when it has not been exactly determined. A comprehensive theoretical treatment is presented in the case of a stationary linear iterative method, where we characterise both the rate of contraction of  mu_m to an atomic measure on x and the nature of the uncertainty quantification being provided. We conclude with an empirical illustration that highlights the potential for probabilistic iterative methods to provide insight into solution uncertainty. 

   more » « less
2. The Framework for Assessing Changes To Sea-level (FACTS) v1.0: a platform for characterizing parametric and structural uncertainty in future global, relative, and extreme sea-level change

   https://doi.org/10.5194/gmd-16-7461-2023  

   Kopp, Robert E.; Garner, Gregory G.; Hermans, Tim H.; Jha, Shantenu; Kumar, Praveen; Reedy, Alexander; Slangen, Aimée B.; Turilli, Matteo; Edwards, Tamsin L.; Gregory, Jonathan M.; et al (December 2023, Geoscientific Model Development)

   Future sea-level rise projections are characterized by both quantifiable uncertainty and unquantifiable structural uncertainty. Thorough scientific assessment of sea-level rise projections requires analysis of both dimensions of uncertainty. Probabilistic sea-level rise projections evaluate the quantifiable dimension of uncertainty; comparison of alternative probabilistic methods provides an indication of structural uncertainty. Here we describe the Framework for Assessing Changes To Sea-level (FACTS), a modular platform for characterizing different probability distributions for the drivers of sea-level change and their consequences for global mean, regional, and extreme sea-level change. We demonstrate its application by generating seven alternative probability distributions under multiple emissions scenarios for both future global mean sea-level change and future relative and extreme sea-level change at New York City. These distributions, closely aligned with those presented in the Intergovernmental Panel on Climate Change Sixth Assessment Report, emphasize the role of the Antarctic and Greenland ice sheets as drivers of structural uncertainty in sea-level change projections. 

   more » « less
3. Learning in the Presence of Low-dimensional Structure: A Spiked Random Matrix Perspective

   Ba, Jimmy; Erdogdu, Murat A; Suzuki, Taiji; Wang, Zhichao; Wu, Denny (December 2023, Advances in Neural Information Processing Systems 36 (NeurIPS 2023))

   Oh, A; Naumann, T; Globerson, A; Saenko, K; Hardt, M; Levine, S (Ed.)

   We consider the problem of learning a single-index target function f∗ : Rd → R under the spiked covariance data: f∗(x) = σ∗   √ 1 1+θ ⟨x,μ⟩   , x ∼ N(0, Id + θμμ⊤), θ ≍ dβ for β ∈ [0, 1), where the link function σ∗ : R → R is a degree-p polynomial with information exponent k (defined as the lowest degree in the Hermite expansion of σ∗), and it depends on the projection of input x onto the spike (signal) direction μ ∈ Rd. In the proportional asymptotic limit where the number of training examples n and the dimensionality d jointly diverge: n, d → ∞, n/d → ψ ∈ (0,∞), we ask the following question: how large should the spike magnitude θ be, in order for (i) kernel methods, (ii) neural networks optimized by gradient descent, to learn f∗? We show that for kernel ridge regression, β ≥ 1 − 1 p is both sufficient and necessary. Whereas for two-layer neural networks trained with gradient descent, β > 1 − 1 k suffices. Our results demonstrate that both kernel methods and neural networks benefit from low-dimensional structures in the data. Further, since k ≤ p by definition, neural networks can adapt to such structures more effectively. 

   more » « less
4. Bayesian Probabilistic Inference of Nonparametric Distance Distributions in DEER Spectroscopy

   https://doi.org/10.1021/acs.jpca.4c05056  

   Sweger, Sarah R; Cheung, Julian C; Zha, Lukas; Pribitzer, Stephan; Stoll, Stefan (October 2024, The Journal of Physical Chemistry A)

   Double electron−electron resonance (DEER) spectroscopy measures distance distributions between spin labels in proteins, yielding important structural and energetic information about conformational landscapes. Analysis of an experimental DEER signal in terms of a distance distribution is a nontrivial task due to the ill-posed nature of the underlying mathematical inversion problem. This work introduces a Bayesian probabilistic inference approach to analyze DEER data, assuming a nonparametric distance distribution with a Tikhonov smoothness prior. The method uses Markov Chain Monte Carlo sampling with a compositional Gibbs sampler to determine a posterior probability distribution over the entire parameter space, including the distance distribution, given an experimental data set. This posterior contains all of the information available from the data, including a full quantification of the uncertainty about the model parameters. The corresponding uncertainty about the distance distribution is visually captured via an ensemble of posterior predictive distributions. Several examples are presented to illustrate the method. Compared with bootstrapping, it performs faster and provides slightly larger uncertainty intervals. 

   more » « less
5. Belief propagation for networks with loops

   https://doi.org/10.1126/sciadv.abf1211  

   Kirkley, Alec; Cantwell, George T.; Newman, M. E. (April 2021, Science Advances)

   null (Ed.)

   Belief propagation is a widely used message passing method for the solution of probabilistic models on networks such as epidemic models, spin models, and Bayesian graphical models, but it suffers from the serious shortcoming that it works poorly in the common case of networks that contain short loops. Here, we provide a solution to this long-standing problem, deriving a belief propagation method that allows for fast calculation of probability distributions in systems with short loops, potentially with high density, as well as giving expressions for the entropy and partition function, which are notoriously difficult quantities to compute. Using the Ising model as an example, we show that our approach gives excellent results on both real and synthetic networks, improving substantially on standard message passing methods. We also discuss potential applications of our method to a variety of other problems. 

   more » « less