skip to main content


This content will become publicly available on April 26, 2025

Title: Multi-fidelity Bayesian Optimization with Multiple Information Sources of Input-dependent Fidelity
By querying approximate surrogate models of different fidelity as available information sources, Multi-Fidelity Bayesian Optimization (MFBO) aims at optimizing unknown functions that are costly or infeasible to evaluate. Existing MFBO methods often assume that approximate surrogates have consistently high or low fidelity across the input domain. However, approximate evaluations from the same surrogate can have different fidelity at different input regions due to data availability and model constraints, especially when considering machine learning surrogates. In this work, we investigate MFBO when multi-fidelity approximations have input-dependent fidelity. By explicitly capturing input dependency for multi-fidelity queries in a Gaussian Process (GP), our new input-dependent MFBO (iMFBO) with learnable noise models better captures the fidelity of each information source in an intuitive way. We further design a new acquisition function for iMFBO and prove that the queries selected by iMFBO have higher quality than those by naive MFBO methods, with a derived sub-linear regret bound. Experiments on both synthetic and real-world data demonstrate its superior empirical performance.  more » « less
Award ID(s):
2119103
PAR ID:
10542584
Author(s) / Creator(s):
; ; ; ; ; ;
Publisher / Repository:
UAI 2024 Conference
Date Published:
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Highly nonlinear dynamic finite element simulations using explicit time integration are particularly valuable tools for structural analysis in fields like automotive, aerospace, and civil engineering, or in the study of injury biomechanics. However, such state-of-the-art simulation models demand significant computational resources. Conventional data-driven surrogate modeling approaches address this by evolving the dynamics on low-dimensional embeddings, yet the majority of them operate directly on high-resolution data obtained from numerical discretizations, making them costly and unsuitable for adaptive resolutions or for handling information flow over large spatial distances. We therefore propose a multi-hierarchical framework for the structured creation of a series of surrogate models at different resolutions. Macroscale features are captured on coarse surrogates, while microscale effects are resolved on finer ones, while leveraging transfer learning to pass information between scales. The objective of this study is to develop efficient surrogates for a kart frame model in a frontal impact scenario. To achieve this, its mesh is simplified to obtain multi-resolution representations of the kart. Subsequently, a graph-convolutional neural network-based surrogate learns parameter-dependent low-dimensional latent dynamics on the coarsest representation. Following surrogates are trained on residuals using finer resolutions, allowing for multiple surrogates with varying hardware requirements and increasing accuracy.

     
    more » « less
  2. Chaudhuri, K (Ed.)
    How can we efficiently gather information to optimize an unknown function, when presented with multiple, mutually dependent information sources with different costs? For example, when optimizing a physical system, intelligently trading off computer simulations and real-world tests can lead to significant savings. Existing multi-fidelity Bayesian optimization methods, such as multi-fidelity GP-UCB or Entropy Search-based approaches, either make simplistic assumptions on the interaction among different fidelities or use simple heuristics that lack theoretical guarantees. In this paper, we study multifidelity Bayesian optimization with complex structural dependencies among multiple outputs, and propose MF-MI-Greedy, a principled algorithmic framework for addressing this problem. In particular, we model different fidelities using additive Gaussian processes based on shared latent relationships with the target function. Then we use cost-sensitive mutual information gain for efficient Bayesian optimization. We propose a simple notion of regret which incorporates the varying cost of different fidelities, and prove that MF-MI-Greedy achieves low regret. We demonstrate the strong empirical performance of our algorithm on both synthetic and real-world datasets. 
    more » « less
  3. Abstract We present methods for emulating the matter power spectrum by combining information from cosmological N-body simulations at different resolutions. An emulator allows estimation of simulation output by interpolating across the parameter space of a limited number of simulations. We present the first implementation in cosmology of multi-fidelity emulation, where many low-resolution simulations are combined with a few high-resolution simulations to achieve an increased emulation accuracy. The power spectrum’s dependence on cosmology is learned from the low-resolution simulations, which are in turn calibrated using high-resolution simulations. We show that our multi-fidelity emulator predicts high-fidelity counterparts to percent-level relative accuracy when using only 3 high-fidelity simulations and outperforms a single-fidelity emulator that uses 11 simulations, although we do not attempt to produce a converged emulator with high absolute accuracy. With a fixed number of high-fidelity training simulations, we show that our multi-fidelity emulator is ≃ 100 times better than a single-fidelity emulator at k ≤ 2 hMpc−1, and ≃ 20 times better at 3 ≤ k < 6.4 hMpc−1. Multi-fidelity emulation is fast to train, using only a simple modification to standard Gaussian processes. Our proposed emulator shows a new way to predict non-linear scales by fusing simulations from different fidelities. 
    more » « less
  4. Designing and/or controlling complex systems in science and engineering relies on appropriate mathematical modeling of systems dynamics. Classical differential equation based solutions in applied and computational mathematics are often computationally demanding. Recently, the connection between reduced-order models of high-dimensional differential equation systems and surrogate machine learning models has been explored. However, the focus of both existing reduced-order and machine learning models for complex systems has been how to best approximate the high fidelity model of choice. Due to high complexity and often limited training data to derive reduced-order or machine learning surrogate models, it is critical for derived reduced-order models to have reliable uncertainty quantification at the same time. In this paper, we propose such a novel framework of Bayesian reduced-order models naturally equipped with uncertainty quantification as it learns the distributions of the parameters of the reduced-order models instead of their point estimates. In particular, we develop learnable Bayesian proper orthogonal decomposition (BayPOD) that learns the distributions of both the POD projection bases and the mapping from the system input parameters to the projected scores/coefficients so that the learned BayPOD can help predict high-dimensional systems dynamics/fields as quantities of interest in different setups with reliable uncertainty estimates. The developed learnable BayPOD inherits the capability of embedding physics constraints when learning the POD-based surrogate reduced-order models, a desirable feature when studying complex systems in science and engineering applications where the available training data are limited. Furthermore, the proposed BayPOD method is an end-to-end solution, which unlike other surrogate-based methods, does not require separate POD and machine learning steps. The results from a real-world case study of the pressure field around an airfoil. 
    more » « less
  5. Programmers and researchers are increasingly developing surrogates of programs, models of a subset of the observable behavior of a given program, to solve a variety of software development challenges. Programmers train surrogates from measurements of the behavior of a program on a dataset of input examples. A key challenge of surrogate construction is determining what training data to use to train a surrogate of a given program.

    We present a methodology for sampling datasets to train neural-network-based surrogates of programs. We first characterize the proportion of data to sample from each region of a program's input space (corresponding to different execution paths of the program) based on the complexity of learning a surrogate of the corresponding execution path. We next provide a program analysis to determine the complexity of different paths in a program. We evaluate these results on a range of real-world programs, demonstrating that complexity-guided sampling results in empirical improvements in accuracy.

     
    more » « less