More Like this

1. MFBO-SSM: Multi-Fidelity Bayesian Optimization for Fast Inference in State-Space Models

   Imani, M; Ghoreishi, S.F.; Allaire, D.; Braga-Neto, U (July 2019, Proceedings of the ... AAAI Conference on Artificial Intelligence)

   Nonlinear state-space models are ubiquitous in modeling real-world dynamical systems. Sequential Monte Carlo (SMC) techniques, also known as particle methods, are a well-known class of parameter estimation methods for this general class of state-space models. Existing SMC-based techniques rely on excessive sampling of the parameter space, which makes their computation intractable for large systems or tall data sets. Bayesian optimization techniques have been used for fast inference in state-space models with intractable likelihoods. These techniques aim to find the maximum of the likelihood function by sequential sampling of the parameter space through a single SMC approximator. Various SMC approximators with different fidelities and computational costs are often available for sample- based likelihood approximation. In this paper, we propose a multi-fidelity Bayesian optimization algorithm for the inference of general nonlinear state-space models (MFBO-SSM), which enables simultaneous sequential selection of parameters and approximators. The accuracy and speed of the algorithm are demonstrated by numerical experiments using synthetic gene expression data from a gene regulatory network model and real data from the VIX stock price index. 

   more » « less
2. Analysis of random sequential message passing algorithms for approximate inference

   https://doi.org/10.1088/1742-5468/ac764a  

   Çakmak, Burak; Lu, Yue M; Opper, Manfred (July 2022, Journal of Statistical Mechanics: Theory and Experiment)

   Abstract We analyze the dynamics of a random sequential message passing algorithm for approximate inference with large Gaussian latent variable models in a student–teacher scenario. To model nontrivial dependencies between the latent variables, we assume random covariance matrices drawn from rotation invariant ensembles. Moreover, we consider a model mismatching setting, where the teacher model and the one used by the student may be different. By means of dynamical functional approach, we obtain exact dynamical mean-field equations characterizing the dynamics of the inference algorithm. We also derive a range of model parameters for which the sequential algorithm does not converge. The boundary of this parameter range coincides with the de Almeida Thouless (AT) stability condition of the replica-symmetric ansatz for the static probabilistic model. 

   more » « less
3. On the Effects of Type II Left Censoring in Stable and Chaotic Compartmental Models for Infectious Diseases: Do Small Sample Estimates Survive Censoring?

   https://doi.org/10.1609/aaaiss.v2i1.27715  

   Selvitella, Alessandro Maria; Foster, Kathleen Lois (January 2024, Proceedings of the AAAI Symposium Series)

   In this paper, we discuss a selection of tools from dynamical systems and order statistics, which are most often utilized separately, and combine them into an algorithm to estimate the parameters of mathematical models for infectious diseases in the case of small sample sizes and left censoring, which is relevant in the case of rapidly evolving infectious diseases and remote populations. The proposed method relies on the analogy between survival functions and the dynamics of the susceptible compartment in SIR-type models, which are both monotone decreasing in time and are both determined by a dual variable: the hazard function in survival prediction and the number of infected people in SIR-type models. We illustrate the methodology in the case of a continuous model in the presence of noisy measurements with different distributions (Normal, Poisson, Negative Binomial) and in a discrete model, reminiscent of the Ricker map, which admits chaotic dynamics. This estimation procedure shows stable results in experiments based on a popular benchmark dataset for SIR-type models and small samples. This manuscript illustrates how classical theoretical statistical methods and dynamical systems can be merged in interesting ways to study problems ranging from more fundamental small sample situations to more complex infectious disease and survival models, with the potential that this tools can be applied in the presence of a large number of covariates and different types of censored data. 

   more » « less
4. Bayesian Recurrent Neural Network Models for Forecasting and Quantifying Uncertainty in Spatial-Temporal Data

   https://doi.org/10.3390/e21020184  

   McDermott, Patrick; Wikle, Christopher (February 2019, Entropy)

   Recurrent neural networks (RNNs) are nonlinear dynamical models commonly used in the machine learning and dynamical systems literature to represent complex dynamical or sequential relationships between variables. Recently, as deep learning models have become more common, RNNs have been used to forecast increasingly complicated systems. Dynamical spatio-temporal processes represent a class of complex systems that can potentially benefit from these types of models. Although the RNN literature is expansive and highly developed, uncertainty quantification is often ignored. Even when considered, the uncertainty is generally quantified without the use of a rigorous framework, such as a fully Bayesian setting. Here we attempt to quantify uncertainty in a more formal framework while maintaining the forecast accuracy that makes these models appealing, by presenting a Bayesian RNN model for nonlinear spatio-temporal forecasting. Additionally, we make simple modifications to the basic RNN to help accommodate the unique nature of nonlinear spatio-temporal data. The proposed model is applied to a Lorenz simulation and two real-world nonlinear spatio-temporal forecasting applications. 

   more » « less
5. Statistical Optimality and Stability of Tangent Transform Algorithms in Logit Models

   Ghosh, I. (July 2022, Journal of machine learning research)

   Pierre Alquier (Ed.)

   A systematic approach to finding variational approximation in an otherwise intractable non-conjugate model is to exploit the general principle of convex duality by minorizing the marginal likelihood that renders the problem tractable. While such approaches are popular in the context of variational inference in non-conjugate Bayesian models, theoretical guarantees on statistical optimality and algorithmic convergence are lacking. Focusing on logistic regression models, we provide mild conditions on the data generating process to derive non-asymptotic upper bounds to the risk incurred by the variational optima. We demonstrate that these assumptions can be completely relaxed if one considers a slight variation of the algorithm by raising the likelihood to a fractional power. Next, we utilize the theory of dynamical systems to provide convergence guarantees for such algorithms in logistic and multinomial logit regression. In particular, we establish local asymptotic stability of the algorithm without any assumptions on the data-generating process. We explore a special case involving a semi-orthogonal design under which a global convergence is obtained. The theory is further illustrated using several numerical studies. 

   more » « less