More Like this

1. Epistemic Uncertainty Quantification in State-space LPV Model Identification Using Bayesian Neural Networks

   Bao, Yajie; Mohammadpour Velni, Javad; Shahbakhti, Mahdi (January 2020, IEEE control systems letters)

   This paper presents a variational Bayesian inference Neural Network (BNN) approach to quantify uncertainties in matrix function estimation for the state-space linear parameter-varying (LPV) model identification problem using only inputs/outputs data. The proposed method simultaneously estimates states and posteriors of matrix functions given data. In particular, states are estimated by reaching a consensus between an estimator based on past system trajectory and an estimator by recurrent equations of states; posteriors are approximated by minimizing the Kullback–Leibler (KL) divergence between the parameterized posterior distribution and the true posterior of the LPV model parameters. Furthermore, techniques such as transfer learning are explored in this work to reduce computational cost and prevent convergence failure of Bayesian inference. The proposed data-driven method is validated using experimental data for identification of a control-oriented reactivity controlled compression ignition (RCCI) engine model. 

   more » « less
2. Diverse and faithful knowledge-grounded dialogue generation via sequential posterior inference

   Xu, Y.; Kong, D.; Xu, D.; Ji, Z.; Pang, B.; Fung, P.; Wu, Y. N. (October 2023, International Conference on Machine Learning (ICML 2023))

   The capability to generate responses with diversity and faithfulness using factual knowledge is paramount for creating a human-like, trustworthy dialogue system. Common strategies either adopt a two-step paradigm, which optimizes knowledge selection and response generation separately and may overlook the inherent correlation between these two tasks, or leverage conditional variational method to jointly optimize knowledge selection and response generation by employing an inference network. In this paper, we present an end-to-end learning framework, termed Sequential Posterior Inference (SPI), capable of se- lecting knowledge and generating dialogues by approximately sampling from the posterior distribution. Unlike other methods, SPI does not require the inference network or assume a simple geometry of the posterior distribution. This straightforward and intuitive inference procedure of SPI directly queries the response generation model, allowing for accurate knowledge selection and generation of faithful responses. In addition to modeling contributions, our experimental results on two common dialogue datasets (Wizard of Wikipedia and Holl-E) demonstrate that SPI outperforms previous strong baselines according to both automatic and human evaluation metrics. 

   more » « less
3. Rapid Model Comparison by Amortizing Across Models

   Zhang, Lily H.; Hughes, Michael C. (December 2019, Proceedings of The 2nd Symposium on Advances in Approximate Bayesian Inference)

   Comparing the inferences of diverse candidate models is an essential part of model checking and escaping local optima. To enable efficient comparison, we introduce an amortized variational inference framework that can perform fast and reliable posterior estimation across models of the same architecture. Our Any Parameter Encoder (APE) extends the encoder neural network common in amortized inference to take both a data feature vector and a model parameter vector as input. APE thus reduces posterior inference across unseen data and models to a single forward pass. In experiments comparing candidate topic models for synthetic data and product reviews, our Any Parameter Encoder yields comparable posteriors to more expensive methods in far less time, especially when the encoder architecture is designed in model-aware fashion. 

   more » « less
4. Rapid Model Comparison by Amortizing Across Models

   Zhang, Lily H.; Hughes, Michael C. (December 2019, Proceedings of The 2nd Symposium on Advances in Approximate Bayesian Inference)

   Comparing the inferences of diverse candidate models is an essential part of model checking and escaping local optima. To enable efficient comparison, we introduce an amortized variational inference framework that can perform fast and reliable posterior estimation across models of the same architecture. Our Any Parameter Encoder (APE) extends the encoder neural network common in amortized inference to take both a data feature vector and a model parameter vector as input. APE thus reduces posterior inference across unseen data and models to a single forward pass. In experiments comparing candidate topic models for synthetic data and product reviews, our Any Parameter Encoder yields comparable posteriors to more expensive methods in far less time, especially when the encoder architecture is designed in model-aware fashion. 

   more » « less
5. Streaming Variational Monte Carlo

   https://doi.org/10.1109/TPAMI.2022.3153225  

   Zhao, Yuan; Nassar, Josue; Jordan, Ian; Bugallo, Monica; Park, Il Memming (February 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence)

   Nonlinear state-space models are powerful tools to describe dynamical structures in complex time series. In a streaming setting where data are processed one sample at a time, simultaneous inference of the state and its nonlinear dynamics has posed significant challenges in practice. We develop a novel online learning framework, leveraging variational inference and sequential Monte Carlo, which enables flexible and accurate Bayesian joint filtering. Our method provides an approximation of the filtering posterior which can be made arbitrarily close to the true filtering distribution for a wide class of dynamics models and observation models. Specifically, the proposed framework can efficiently approximate a posterior over the dynamics using sparse Gaussian processes, allowing for an interpretable model of the latent dynamics. Constant time complexity per sample makes our approach amenable to online learning scenarios and suitable for real-time applications. 

   more » « less