More Like this

1. A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes

   Song, JL (April 2019, Proceedings of Machine Learning Research)

   Chaudhuri, K (Ed.)

   How can we efficiently gather information to optimize an unknown function, when presented with multiple, mutually dependent information sources with different costs? For example, when optimizing a physical system, intelligently trading off computer simulations and real-world tests can lead to significant savings. Existing multi-fidelity Bayesian optimization methods, such as multi-fidelity GP-UCB or Entropy Search-based approaches, either make simplistic assumptions on the interaction among different fidelities or use simple heuristics that lack theoretical guarantees. In this paper, we study multifidelity Bayesian optimization with complex structural dependencies among multiple outputs, and propose MF-MI-Greedy, a principled algorithmic framework for addressing this problem. In particular, we model different fidelities using additive Gaussian processes based on shared latent relationships with the target function. Then we use cost-sensitive mutual information gain for efficient Bayesian optimization. We propose a simple notion of regret which incorporates the varying cost of different fidelities, and prove that MF-MI-Greedy achieves low regret. We demonstrate the strong empirical performance of our algorithm on both synthetic and real-world datasets. 

   more » « less
2. Multi-fidelity Bayesian Optimization with Multiple Information Sources of Input-dependent Fidelity

   Fan, Mingzhou; Yoon, Byung-Jun; Dougherty, Edward; Urban, Nathan; Alexander, Francis; Arróyave, Raymundo; Qian, Xiaoning (April 2024, UAI 2024 Conference)

   By querying approximate surrogate models of different fidelity as available information sources, Multi-Fidelity Bayesian Optimization (MFBO) aims at optimizing unknown functions that are costly or infeasible to evaluate. Existing MFBO methods often assume that approximate surrogates have consistently high or low fidelity across the input domain. However, approximate evaluations from the same surrogate can have different fidelity at different input regions due to data availability and model constraints, especially when considering machine learning surrogates. In this work, we investigate MFBO when multi-fidelity approximations have input-dependent fidelity. By explicitly capturing input dependency for multi-fidelity queries in a Gaussian Process (GP), our new input-dependent MFBO (iMFBO) with learnable noise models better captures the fidelity of each information source in an intuitive way. We further design a new acquisition function for iMFBO and prove that the queries selected by iMFBO have higher quality than those by naive MFBO methods, with a derived sub-linear regret bound. Experiments on both synthetic and real-world data demonstrate its superior empirical performance. 

   more » « less
3. Mitigating the Effects of Source-Dependent Bias and Noise on Multi-Source Bayesian Optimization: Application to Materials Design

   https://doi.org/10.1115/DETC2023-114414  

   Zanjani_Foumani, Zahra; Yousefpour, Amin; Shishehbor, Mehdi; Bostanabad, Ramin (August 2023, American Society of Mechanical Engineers)

   Abstract Bayesian optimization (BO) is a sequential optimization strategy that is increasingly employed in a wide range of areas including materials design. In real world applications, acquiring high-fidelity (HF) data through physical experiments or HF simulations is the major cost component of BO. To alleviate this bottleneck, multi-fidelity (MF) methods are increasingly used to forgo the sole reliance on the expensive HF data and reduce the sampling costs by querying inexpensive low-fidelity (LF) sources whose data are correlated with HF samples. Existing multi-fidelity BO (MFBO) methods operate under the following two assumptions: (1) Leveraging global (rather than local) correlation between HF and LF sources, and (2) Associating all the data sources with the same noise process. These assumptions dramatically reduce the performance of MFBO when LF sources are only locally correlated with the HF source or when the noise variance varies across the data sources. To dispense with these incorrect assumptions, we propose an MF emulation method that (1) learns a noise model for each data source, and (2) enables BO to leverage highly biased LF sources which are only locally correlated with the HF source. We illustrate the performance of our method through analytical examples and engineering problems on materials design. 

   more » « less
4. Cross-Referenced Dictionaries and the Limits of Write Optimization

   https://doi.org/10.1137/1.9781611974782.99  

   Afshani, Peyman; Bender, Michael; Farach-Colton, Martin; Fineman, Jeremy; Goswami, Mayank; Tsai, Meng-Tsung Tsai (January 2017, Proceedings of the Twenty-Eighth Annual ACM-SIAM Symposium on Discrete Algorithms)

   Dictionaries remain the most well studied class of data structures. A dictionary supports insertions, deletions, membership queries, and usually successor, predecessor, and extract-min. In a RAM, all such operations take O(log n) time on n elements. Dictionaries are often cross-referenced as follows. Consider a set of tuples {〈ai,bi,ci…〉}. A database might include more than one dictionary on such a set, for example, one indexed on the a ‘s, another on the b‘s, and so on. Once again, in a RAM, inserting into a set of L cross-referenced dictionaries takes O(L log n) time, as does deleting. The situation is more interesting in external memory. On a Disk Access Machine (DAM), B-trees achieve O(logB N) I/Os for insertions and deletions on a single dictionary and K-element range queries take optimal O(logB N + K/B) I/Os. These bounds are also achievable by a B-tree on cross-referenced dictionaries, with a slowdown of an L factor on insertion and deletions. In recent years, both the theory and practice of external- memory dictionaries has been revolutionized by write- optimization techniques. A dictionary is write optimized if it is close to a B-tree for query time while beating B-trees on insertions. The best (and optimal) dictionaries achieve a substantially improved insertion and deletion cost of amortized I/Os on a single dictionary while maintaining optimal O(log1+B∊ N + K/B)- I/O range queries. Although write optimization still helps for insertions into cross-referenced dictionaries, its value for deletions would seem to be greatly reduced. A deletion into a cross- referenced dictionary only specifies a key a. It seems to be necessary to look up the associated values b, c … in order to delete them from the other dictionaries. This takes Ω(logB N) I/Os, well above the per-dictionary write-optimization budget of So the total deletion cost is In short, for deletions, write optimization offers an advantage over B-trees in that L multiplies a lower order term, but when L = 2, write optimization seems to offer no asymptotic advantage over B-trees. That is, no known query- optimal solution for pairs of cross-referenced dictionaries seem to beat B-trees for deletions. In this paper, we show a lower bound establishing that a pair of cross-referenced dictionaries that are optimal for range queries and that supports deletions cannot match the write optimization bound available to insert-only dictionaries. This result thus establishes a limit to the applicability of write-optimization techniques on which many new databases and file systems are based. Read More: http://epubs.siam.org/doi/10.1137/1.9781611974782.99 

   more » « less
5. A Data-Driven Approach for Process Optimization of Metallic Additive Manufacturing Under Uncertainty

   https://doi.org/10.1115/1.4043798  

   Wang, Zhuo; Liu, Pengwei; Xiao, Yaohong; Cui, Xiangyang; Hu, Zhen; Chen, Lei (August 2019, Journal of Manufacturing Science and Engineering)

   The presence of various uncertainty sources in metal-based additive manufacturing (AM) process prevents producing AM products with consistently high quality. Using electron beam melting (EBM) of Ti-6Al-4V as an example, this paper presents a data-driven framework for process parameters optimization using physics-informed computer simulation models. The goal is to identify a robust manufacturing condition that allows us to constantly obtain equiaxed materials microstructures under uncertainty. To overcome the computational challenge in the robust design optimization under uncertainty, a two-level data-driven surrogate model is constructed based on the simulation data of a validated high-fidelity multiphysics AM simulation model. The robust design result, indicating a combination of low preheating temperature, low beam power, and intermediate scanning speed, was acquired enabling the repetitive production of equiaxed structure products as demonstrated by physics-based simulations. Global sensitivity analysis at the optimal design point indicates that among the studied six noise factors, specific heat capacity and grain growth activation energy have the largest impact on the microstructure variation. Through this exemplar process optimization, the current study also demonstrates the promising potential of the presented approach in facilitating other complicate AM process optimizations, such as robust designs in terms of porosity control or direct mechanical property control. 

   more » « less