More Like this

1. Novel Tool for Selecting Surrogate Modeling Techniques for Surface Approximation

   https://doi.org/10.1016/B978-0-323-88506-5.50071-1  

   Williams, B. Cremaschi (January 2021, Computer aided chemical engineering)

   Turkay, M. Aydin (Ed.)

   Surrogate models are used to map input data to output data when the actual relationship between the two is unknown or computationally expensive to evaluate for several applications, including surface approximation and surrogate-based optimization. Many techniques have been developed for surrogate modeling; however, a systematic method for selecting suitable techniques for an application remains an open challenge. This work compares the performance of eight surrogate modeling techniques for approximating a surface over a set of simulated data. Using the comparison results, we constructed a Random Forest based tool to recommend the appropriate surrogate modeling technique for a given dataset using attributes calculated only from the available input and output values. The tool identifies the appropriate surrogate modeling techniques for surface approximation with an accuracy of 87% and a precision of 86%. Using the tool for surrogate model form selection enables computational time savings by avoiding expensive trial-and-error selection methods. 

   more » « less
2. Bayesian optimization with adaptive surrogate models for automated experimental design

   https://doi.org/10.1038/s41524-021-00662-x  

   Lei, Bowen; Kirk, Tanner Quinn; Bhattacharya, Anirban; Pati, Debdeep; Qian, Xiaoning; Arroyave, Raymundo; Mallick, Bani K. (December 2021, npj Computational Materials)

   Abstract Bayesian optimization (BO) is an indispensable tool to optimize objective functions that either do not have known functional forms or are expensive to evaluate. Currently, optimal experimental design is always conducted within the workflow of BO leading to more efficient exploration of the design space compared to traditional strategies. This can have a significant impact on modern scientific discovery, in particular autonomous materials discovery, which can be viewed as an optimization problem aimed at looking for the maximum (or minimum) point for the desired materials properties. The performance of BO-based experimental design depends not only on the adopted acquisition function but also on the surrogate models that help to approximate underlying objective functions. In this paper, we propose a fully autonomous experimental design framework that uses more adaptive and flexible Bayesian surrogate models in a BO procedure, namely Bayesian multivariate adaptive regression splines and Bayesian additive regression trees. They can overcome the weaknesses of widely used Gaussian process-based methods when faced with relatively high-dimensional design space or non-smooth patterns of objective functions. Both simulation studies and real-world materials science case studies demonstrate their enhanced search efficiency and robustness. 

   more » « less
3. Data‐driven decision‐focused surrogate modeling

   https://doi.org/10.1002/aic.18338  

   Gupta, Rishabh; Zhang, Qi (January 2024, AIChE Journal)

   Abstract We introduce the concept of decision‐focused surrogate modeling for solving computationally challenging nonlinear optimization problems in real‐time settings. The proposed data‐driven framework seeks to learn a simpler, for example, convex, surrogate optimization model that is trained to minimize thedecision prediction error, which is defined as the difference between the optimal solutions of the original and the surrogate optimization models. The learning problem, formulated as a bilevel program, can be viewed as a data‐driven inverse optimization problem to which we apply a decomposition‐based solution algorithm from previous work. We validate our framework through numerical experiments involving the optimization of common nonlinear chemical processes such as chemical reactors, heat exchanger networks, and material blending systems. We also present a detailed comparison of decision‐focused surrogate modeling with standard data‐driven surrogate modeling methods and demonstrate that our approach is significantly more data‐efficient while producing simple surrogate models with high decision prediction accuracy. 

   more » « less
4. Toward Reusable Surrogate Models: Graph-Based Transfer Learning on Trusses

   https://doi.org/10.1115/1.4052298  

   Whalen, Eamon; Mueller, Caitlin (February 2022, Journal of Mechanical Design)

   Abstract Surrogate models have several uses in engineering design, including speeding up design optimization, noise reduction, test measurement interpolation, gradient estimation, portability, and protection of intellectual property. Traditionally, surrogate models require that all training data conform to the same parametrization (e.g., design variables), limiting design freedom and prohibiting the reuse of historical data. In response, this article proposes graph-based surrogate models (GSMs) for trusses. The GSM can accurately predict displacement fields from static loads given the structure’s geometry as input, enabling training across multiple parametrizations. GSMs build upon recent advancements in geometric deep learning, which have led to the ability to learn on undirected graphs: a natural representation for trusses. To further promote flexible surrogate models, this article explores transfer learning within the context of engineering design and demonstrates positive knowledge transfer across data sets of different topologies, complexities, loads, and applications, resulting in more flexible and data-efficient surrogate models for trusses. 

   more » « less
5. Variational inference with NoFAS: Normalizing flow with adaptive surrogate for computationally expensive models

   https://doi.org/https://doi.org/10.48550/arXiv.2108.12657  

   Wang, Yu; Liu, Fang; Schiavazzi, Daniele E. (October 2022, Journal of computational physics)

   Fast inference of numerical model parameters from data is an important prerequisite to generate predictive models for a wide range of applications. Use of sampling-based approaches such as Markov chain Monte Carlo may become intractable when each likelihood evaluation is computationally expensive. New approaches combining variational inference with normalizing flow are characterized by a computational cost that grows only linearly with the dimensionality of the latent variable space, and rely on gradient-based optimization instead of sampling, providing a more efficient approach for Bayesian inference about the model parameters. Moreover, the cost of frequently evaluating an expensive likelihood can be mitigated by replacing the true model with an offline trained surrogate model, such as neural networks. However, this approach might generate significant bias when the surrogate is insufficiently accurate around the posterior modes. To reduce the computational cost without sacrificing inferential accuracy, we propose Normalizing Flow with Adaptive Surrogate (NoFAS), an optimization strategy that alternatively updates the normalizing flow parameters and surrogate model parameters. We also propose an efficient sample weighting scheme for surrogate model training that preserves global accuracy while effectively capturing high posterior density regions. We demonstrate the inferential and computational superiority of NoFAS against various benchmarks, including cases where the underlying model lacks identifiability. The source code and numerical experiments used for this study are available at https://github.com/cedricwangyu/NoFAS. 

   more » « less