skip to main content


Title: Information geometry for multiparameter models: new perspectives on the origin of simplicity
Abstract Complex models in physics, biology, economics, and engineering are often sloppy , meaning that the model parameters are not well determined by the model predictions for collective behavior. Many parameter combinations can vary over decades without significant changes in the predictions. This review uses information geometry to explore sloppiness and its deep relation to emergent theories. We introduce the model manifold of predictions, whose coordinates are the model parameters. Its hyperribbon structure explains why only a few parameter combinations matter for the behavior. We review recent rigorous results that connect the hierarchy of hyperribbon widths to approximation theory, and to the smoothness of model predictions under changes of the control variables. We discuss recent geodesic methods to find simpler models on nearby boundaries of the model manifold—emergent theories with fewer parameters that explain the behavior equally well. We discuss a Bayesian prior which optimizes the mutual information between model parameters and experimental data, naturally favoring points on the emergent boundary theories and thus simpler models. We introduce a ‘projected maximum likelihood’ prior that efficiently approximates this optimal prior, and contrast both to the poor behavior of the traditional Jeffreys prior. We discuss the way the renormalization group coarse-graining in statistical mechanics introduces a flow of the model manifold, and connect stiff and sloppy directions along the model manifold with relevant and irrelevant eigendirections of the renormalization group. Finally, we discuss recently developed ‘intensive’ embedding methods, allowing one to visualize the predictions of arbitrary probabilistic models as low-dimensional projections of an isometric embedding, and illustrate our method by generating the model manifold of the Ising model.  more » « less
Award ID(s):
1753357 1834332
NSF-PAR ID:
10404588
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
Reports on Progress in Physics
Volume:
86
Issue:
3
ISSN:
0034-4885
Page Range / eLocation ID:
035901
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. In this paper, we consider the problem of quantifying parametric uncertainty in classical empirical interatomic potentials (IPs) using both Bayesian (Markov Chain Monte Carlo) and frequentist (profile likelihood) methods. We interface these tools with the Open Knowledgebase of Interatomic Models and study three models based on the Lennard-Jones, Morse, and Stillinger–Weber potentials. We confirm that IPs are typically sloppy, i.e., insensitive to coordinated changes in some parameter combinations. Because the inverse problem in such models is ill-conditioned, parameters are unidentifiable. This presents challenges for traditional statistical methods, as we demonstrate and interpret within both Bayesian and frequentist frameworks. We use information geometry to illuminate the underlying cause of this phenomenon and show that IPs have global properties similar to those of sloppy models from fields, such as systems biology, power systems, and critical phenomena. IPs correspond to bounded manifolds with a hierarchy of widths, leading to low effective dimensionality in the model. We show how information geometry can motivate new, natural parameterizations that improve the stability and interpretation of uncertainty quantification analysis and further suggest simplified, less-sloppy models. 
    more » « less
  2. Abstract

    In this article we review how categorical equivalences are realized by renormalization group flow in physical realizations of stacks, derived categories, and derived schemes. We begin by reviewing the physical realization of sigma models on stacks, as (universality classes of) gauged sigma models, and look in particular at properties of sigma models on gerbes (equivalently, sigma models with restrictions on nonperturbative sectors), and ‘decomposition,’ in which two‐dimensional sigma models on gerbes decompose into disjoint unions of ordinary theories. We also discuss stack structures on examples of moduli spaces of SCFTs, focusing on elliptic curves, and implications of subtleties there for string dualities in other dimensions. In the second part of this article, we review the physical realization of derived categories in terms of renormalization group flow (time evolution) of combinations of D‐branes, antibranes, and tachyons. In the third part of this article, we review how Landau–Ginzburg models provide a physical realization of derived schemes, and also outline an example of a derived structure on a moduli spaces of SCFTs.

     
    more » « less
  3. Abstract

    For decades, frustrated quantum magnets have been a seed for scientific progress and innovation in condensed matter. As much as the numerical tools for low-dimensional quantum magnetism have thrived and improved in recent years due to breakthroughs inspired by quantum information and quantum computation, higher-dimensional quantum magnetism can be considered as the final frontier, where strong quantum entanglement, multiple ordering channels, and manifold ways of paramagnetism culminate. At the same time, efforts in crystal synthesis have induced a significant increase in the number of tangible frustrated magnets which are generically three-dimensional in nature, creating an urgent need for quantitative theoretical modeling. We review the pseudo-fermion (PF) and pseudo-Majorana (PM) functional renormalization group (FRG) and their specific ability to address higher-dimensional frustrated quantum magnetism. First developed more than a decade ago, the PFFRG interprets a Heisenberg model Hamiltonian in terms of Abrikosov pseudofermions, which is then treated in a diagrammatic resummation scheme formulated as a renormalization group flow ofm-particle pseudofermion vertices. The article reviews the state of the art of PFFRG and PMFRG and discusses their application to exemplary domains of frustrated magnetism, but most importantly, it makes the algorithmic and implementation details of these methods accessible to everyone. By thus lowering the entry barrier to their application, we hope that this review will contribute towards establishing PFFRG and PMFRG as the numerical methods for addressing frustrated quantum magnetism in higher spatial dimensions.

     
    more » « less
  4. Abstract

    Climate change is having significant impacts on Earth’s ecosystems and carbon budgets, and in the Arctic may drive a shift from an historic carbon sink to a source. Large uncertainties in terrestrial biosphere models (TBMs) used to forecast Arctic changes demonstrate the challenges of determining the timing and extent of this possible switch. This spread in model predictions can limit the ability of TBMs to guide management and policy decisions. One of the most influential sources of model uncertainty is model parameterization. Parameter uncertainty results in part from a mismatch between available data in databases and model needs. We identify that mismatch for three TBMs, DVM-DOS-TEM, SIPNET and ED2, and four databases with information on Arctic and boreal above- and belowground traits that may be applied to model parametrization. However, focusing solely on such data gaps can introduce biases towards simple models and ignores structural model uncertainty, another main source for model uncertainty. Therefore, we develop a causal loop diagram (CLD) of the Arctic and boreal ecosystem that includes unquantified, and thus unmodeled, processes. We map model parameters to processes in the CLD and assess parameter vulnerability via the internal network structure. One important substructure, feed forward loops (FFLs), describe processes that are linked both directly and indirectly. When the model parameters are data-informed, these indirect processes might be implicitly included in the model, but if not, they have the potential to introduce significant model uncertainty. We find that the parameters describing the impact of local temperature on microbial activity are associated with a particularly high number of FFLs but are not constrained well by existing data. By employing ecological models of varying complexity, databases, and network methods, we identify the key parameters responsible for limited model accuracy. They should be prioritized for future data sampling to reduce model uncertainty.

     
    more » « less
  5. Background

    In systems biology, the dynamics of biological networks are often modeled with ordinary differential equations (ODEs) that encode interacting components in the systems, resulting in highly complex models. In contrast, the amount of experimentally available data is almost always limited, and insufficient to constrain the parameters. In this situation, parameter estimation is a very challenging problem. To address this challenge, two intuitive approaches are to perform experimental design to generate more data, and to perform model reduction to simplify the model. Experimental design and model reduction have been traditionally viewed as two distinct areas, and an extensive literature and excellent reviews exist on each of the two areas. Intriguingly, however, the intrinsic connections between the two areas have not been recognized.

    Results

    Experimental design and model reduction are deeply related, and can be considered as one unified framework. There are two recent methods that can tackle both areas, one based on model manifold and the other based on profile likelihood. We use a simple sum‐of‐two‐exponentials example to discuss the concepts and algorithmic details of both methods, and provide Matlab‐based code and implementation which are useful resources for the dissemination and adoption of experimental design and model reduction in the biology community.

    Conclusions

    From a geometric perspective, we consider the experimental data as a point in a high‐dimensional data space and the mathematical model as a manifold living in this space. Parameter estimation can be viewed as a projection of the data point onto the manifold. By examining the singularity around the projected point on the manifold, we can perform both experimental design and model reduction. Experimental design identifies new experiments that expand the manifold and remove the singularity, whereas model reduction identifies the nearest boundary, which is the nearest singularity that suggests an appropriate form of a reduced model. This geometric interpretation represents one step toward the convergence of experimental design and model reduction as a unified framework.

     
    more » « less