skip to main content


Title: Sensitivity Analysis and Bayesian Calibration of OpenSees Models Using quoFEM
The NHERI SimCenter is a nine-year research project that aims to advance the simulation of natural hazard impact on the built environment and communities. The SimCenter is developing several open-source workflow applications and an underlying scientific application framework. All applications built on this framework provide an OpenSees interface that enables users to use their existing models in advanced simulation studies, such as local and regional performance assessment, and uncertainty quantification (UQ). SimCenter applications provide researchers an opportunity to explore different extensions of their models by lowering the interdisciplinary barrier and encouraging collaboration. Among the applications, quoFEM provides access to UQ analyses with an easy-to-use, standardized interface. This work demonstrates the research enabled by quoFEM through the example of model calibration using PM4Sand, a soil constitutive model available in OpenSees. After an initial sensitivity analysis, the model is calibrated using Bayesian inference based on observations of hysteretic soil response from cyclic direct simple shear tests. The uncertainty in the model parameters is used in forward propagation to explore plausible lateral spreading scenarios due to seismic liquefaction. The results demonstrate the utility of quoFEM to the OpenSees community as a UQ-enabling tool.  more » « less
Award ID(s):
2131111
PAR ID:
10501938
Author(s) / Creator(s):
; ; ; ; ;
Publisher / Repository:
Springer, Cham
Date Published:
Journal Name:
Proceedings of the 2022 Eurasian OpenSees Days
ISBN:
978-3-031-30125-4
Format(s):
Medium: X
Location:
https://doi.org/10.1007/978-3-031-30125-4_6
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Neural networks (NN) have become an important tool for prediction tasks—both regression and classification—in environmental science. Since many environmental-science problems involve life-or-death decisions and policy making, it is crucial to provide not only predictions but also an estimate of the uncertainty in the predictions. Until recently, very few tools were available to provide uncertainty quantification (UQ) for NN predictions. However, in recent years the computer-science field has developed numerous UQ approaches, and several research groups are exploring how to apply these approaches in environmental science. We provide an accessible introduction to six of these UQ approaches, then focus on tools for the next step, namely, to answer the question:Once we obtain an uncertainty estimate (using any approach), how do we know whether it is good or bad?To answer this question, we highlight four evaluation graphics and eight evaluation scores that are well suited for evaluating and comparing uncertainty estimates (NN based or otherwise) for environmental-science applications. We demonstrate the UQ approaches and UQ-evaluation methods for two real-world problems: 1) estimating vertical profiles of atmospheric dewpoint (a regression task) and 2) predicting convection over Taiwan based onHimawari-8satellite imagery (a classification task). We also provide Jupyter notebooks with Python code for implementing the UQ approaches and UQ-evaluation methods discussed herein. This article provides the environmental-science community with the knowledge and tools to start incorporating the large number of emerging UQ methods into their research.

    Significance Statement

    Neural networks are used for many environmental-science applications, some involving life-or-death decision-making. In recent years new methods have been developed to provide much-needed uncertainty estimates for NN predictions. We seek to accelerate the adoption of these methods in the environmental-science community with an accessible introduction to 1) methods for computing uncertainty estimates in NN predictions and 2) methods for evaluating such estimates.

     
    more » « less
  2. Abstract Modeling and simulation for additive manufacturing (AM) are critical enablers for understanding process physics, conducting process planning and optimization, and streamlining qualification and certification. It is often the case that a suite of hierarchically linked (or coupled) simulation models is needed to achieve the above tasks, as the entirety of the complex physical phenomena relevant to the understanding of process-structure-property-performance relationships in the context of AM precludes the use of a single simulation framework. In this study using a Bayesian network approach, we address the important problem of conducting uncertainty quantification (UQ) analysis for multiple hierarchical models to establish process-microstructure relationships in laser powder bed fusion (LPBF) AM. More significantly, we present the framework to calibrate and analyze simulation models that have experimentally unmeasurable variables, which are quantities of interest predicted by an upstream model and deemed necessary for the downstream model in the chain. We validate the framework using a case study on predicting the microstructure of a binary nickel-niobium alloy processed using LPBF as a function of processing parameters. Our framework is shown to be able to predict segregation of niobium with up to 94.3% prediction accuracy on test data. 
    more » « less
  3. Abstract

    Uncertainty quantification (UQ) in metal additive manufacturing (AM) has attracted tremendous interest in order to dramatically improve product reliability. Model-based UQ, which relies on the validity of a computational model, has been widely explored as a potential substitute for the time-consuming and expensive UQ solely based on experiments. However, its adoption in the practical AM process requires overcoming two main challenges: (1) the inaccurate knowledge of uncertainty sources and (2) the intrinsic uncertainty associated with the computational model. Here, we propose a data-driven framework to tackle these two challenges by combining high throughput physical/surrogate model simulations and the AM-Bench experimental data from the National Institute of Standards and Technology (NIST). We first construct a surrogate model, based on high throughput physical simulations, for predicting the three-dimensional (3D) melt pool geometry and its uncertainty with respect to AM parameters and uncertainty sources. We then employ a sequential Bayesian calibration method to perform experimental parameter calibration and model correction to significantly improve the validity of the 3D melt pool surrogate model. The application of the calibrated melt pool model to UQ of the porosity level, an important quality factor, of AM parts, demonstrates its potential use in AM quality control. The proposed UQ framework can be generally applicable to different AM processes, representing a significant advance toward physics-based quality control of AM products.

     
    more » « less
  4. Abstract

    Climate models are generally calibrated manually by comparing selected climate statistics, such as the global top‐of‐atmosphere energy balance, to observations. The manual tuning only targets a limited subset of observational data and parameters. Bayesian calibration can estimate climate model parameters and their uncertainty using a larger fraction of the available data and automatically exploring the parameter space more broadly. In Bayesian learning, it is natural to exploit the seasonal cycle, which has large amplitude compared with anthropogenic climate change in many climate statistics. In this study, we develop methods for the calibration and uncertainty quantification (UQ) of model parameters exploiting the seasonal cycle, and we demonstrate a proof‐of‐concept with an idealized general circulation model (GCM). UQ is performed using the calibrate‐emulate‐sample approach, which combines stochastic optimization and machine learning emulation to speed up Bayesian learning. The methods are demonstrated in a perfect‐model setting through the calibration and UQ of a convective parameterization in an idealized GCM with a seasonal cycle. Calibration and UQ based on seasonally averaged climate statistics, compared to annually averaged, reduces the calibration error by up to an order of magnitude and narrows the spread of the non‐Gaussian posterior distributions by factors between two and five, depending on the variables used for UQ. The reduction in the spread of the parameter posterior distribution leads to a reduction in the uncertainty of climate model predictions.

     
    more » « less
  5. Abstract

    It is critical that machine learning (ML) model predictions be trustworthy for high-throughput catalyst discovery approaches. Uncertainty quantification (UQ) methods allow estimation of the trustworthiness of an ML model, but these methods have not been well explored in the field of heterogeneous catalysis. Herein, we investigate different UQ methods applied to a crystal graph convolutional neural network to predict adsorption energies of molecules on alloys from the Open Catalyst 2020 dataset, the largest existing heterogeneous catalyst dataset. We apply three UQ methods to the adsorption energy predictions, namelyk-fold ensembling, Monte Carlo dropout, and evidential regression. The effectiveness of each UQ method is assessed based on accuracy, sharpness, dispersion, calibration, and tightness. Evidential regression is demonstrated to be a powerful approach for rapidly obtaining tunable, competitively trustworthy UQ estimates for heterogeneous catalysis applications when using neural networks. Recalibration of model uncertainties is shown to be essential in practical screening applications of catalysts using uncertainties.

     
    more » « less