More Like this

1. Learning and Optimization with Bayesian Hybrid Models

   https://doi.org/10.23919/ACC45564.2020.9148007  

   Eugene, E. A; Gao, X.; Dowling, A. W. (July 2021, 2020 American Control Conference (ACC))

   null (Ed.)

   Bayesian hybrid models fuse physics-based insights with machine learning constructs to correct for systematic bias. In this paper, we compare Bayesian hybrid models against physics-based glass-box and Gaussian process black-box surrogate models. We consider ballistic firing as an illustrative case study for a Bayesian decision-making workflow. First, Bayesian calibration is performed to estimate model parameters. We then use the posterior distribution from Bayesian analysis to compute optimal firing conditions to hit a target via a single-stage stochastic program. The case study demonstrates the ability of Bayesian hybrid models to overcome systematic bias from missing physics with fewer data than the pure machine learning approach. Ultimately, we argue Bayesian hybrid models are an emerging paradigm for data-informed decision-making under parametric and epistemic uncertainty. 

   more » « less
2. Learning and optimization under epistemic uncertainty with Bayesian hybrid models

   https://doi.org/10.1016/j.compchemeng.2023.108430  

   Eugene, Elvis A.; Jones, Kyla D.; Gao, Xian; Wang, Jialu; Dowling, Alexander W. (November 2023, Computers & Chemical Engineering)

   Hybrid (i.e., grey-box) models are a powerful and flexible paradigm for predictive science and engineering. Grey-box models use data-driven constructs to incorporate unknown or computationally intractable phenomena into glass-box mechanistic models. The pioneering work of statisticians Kennedy and O’Hagan introduced a new paradigm to quantify epistemic (i.e., model-form) uncertainty. While popular in several engineering disciplines, prior work using Kennedy–O’Hagan hybrid models focuses on prediction with accurate uncertainty estimates. This work demonstrates computational strategies to deploy Bayesian hybrid models for optimization under uncertainty. Specifically, the posterior distributions of Bayesian hybrid models provide a principled uncertainty set for stochastic programming, chance-constrained optimization, or robust optimization. Through two illustrative case studies, we demonstrate the efficacy of hybrid models, composed of a structurally inadequate glass-box model and Gaussian process bias correction term, for decision-making using limited training data. From these case studies, we develop recommended best practices and explore the trade-offs between different hybrid model architectures. 

   more » « less
3. Volatility Based Kernels and Moving Average Means for Accurate Forecasting with Gaussian Processes

   Benton, Gregory; Maddox, Wesley; Wilson, Andrew Gordon (July 2022, Proceedings of the 39th International Conference on Machine Learning)

   A broad class of stochastic volatility models are defined by systems of stochastic differential equations, and while these models have seen widespread success in domains such as finance and statistical climatology, they typically lack an ability to condition on historical data to produce a true posterior distribution. To address this fundamental limitation, we show how to re-cast a class of stochastic volatility models as a hierarchical Gaussian process (GP) model with specialized covariance functions. This GP model retains the inductive biases of the stochastic volatility model while providing the posterior predictive distribution given by GP inference. Within this framework, we take inspiration from well studied domains to introduce a new class of models, Volt and Magpie, that significantly outperform baselines in stock and wind speed forecasting, and naturally extend to the multitask setting. 

   more » « less
4. Spatial and Temporal Bayesian Hierarchical Model Over Large Domains With Application to Holocene Sea Surface Temperature Reconstruction in the Equatorial Pacific

   https://doi.org/10.1029/2024PA004844  

   Ossandón, Álvaro; Gual, Javier; Rajagopalan, Balaji; Kleiber, William; Marchitto, Thomas (December 2024, Paleoceanography and Paleoclimatology)

   We present a novel space‐time Bayesian hierarchical model (BHM) to reconstruct annual Sea Surface Temperature (SST) over a large domain based on SST at limited proxy (i.e., sediment core) locations. The model is tested in the equatorial Pacific. The BHM leverages Principal Component Analysis to identify dominant space‐time modes of contemporary variability of the SST field at the proxy locations and employs these modes in a Gaussian process framework to estimate SSTs across the entire domain. The BHM allows us to model the mean field and covariance, varying in space and time in the process layers of the hierarchy. Using the Markov Chain Monte Carlo (MCMC) method and suitable priors on the model parameters, posterior distributions of the model parameters and, consequently, posterior distributions of the SST fields and the attendant uncertainties are obtained for any desired year. The BHM is calibrated and validated in the contemporary period (1854–2014) and subsequently applied to reconstruct SST fields during the Holocene (0–10 ka). Results are consistent with prior inferences of La Niña‐like conditions during the Holocene. This modeling framework opens exciting prospects for modeling and reconstruction of other fields, such as precipitation, drought indices, and vegetation. 

   more » « less
5. BayGEN: A Bayesian Space‐Time Stochastic Weather Generator

   https://doi.org/10.1029/2017WR022473  

   Verdin, Andrew; Rajagopalan, Balaji; Kleiber, William; Podestá, Guillermo; Bert, Federico (April 2019, Water Resources Research)

   Abstract We present a Bayesian hierarchical space‐time stochastic weather generator (BayGEN) to generate daily precipitation and minimum and maximum temperatures. BayGEN employs a hierarchical framework with data, process, and parameter layers. In the data layer, precipitation occurrence at each site is modeled using probit regression using a spatially distributed latent Gaussian process; precipitation amounts are modeled as gamma random variables; and minimum and maximum temperatures are modeled as realizations from Gaussian processes. The latent Gaussian process that drives the precipitation occurrence process is modeled in the process layer. In the parameter layer, the model parameters of the data and process layers are modeled as spatially distributed Gaussian processes, consequently enabling the simulation of daily weather at arbitrary (unobserved) locations or on a regular grid. All model parameters are endowed with weakly informative prior distributions. The No‐U Turn sampler, an adaptive form of Hamiltonian Monte Carlo, is used to maximize the model likelihood function and obtain posterior samples of each parameter. Posterior samples of the model parameters propagate uncertainty to the weather simulations, an important feature that makes BayGEN unique compared to traditional weather generators. We demonstrate the utility of BayGEN with application to daily weather generation in a basin of the Argentine Pampas. Furthermore, we evaluate the implications of crop yield by driving a crop simulation model with weather simulations from BayGEN and an equivalent non‐Bayesian weather generator. 

   more » « less