More Like this

1. On Empirical Bayes Variational Autoencoder: An Excess Risk Bound

   Tang, Rong; Yang, Yun. (January 2021, Proceedings of Machine Learning Research)

   null (Ed.)

   In this paper, we consider variational autoencoders (VAE) via empirical Bayes estimation, referred to as Empirical Bayes Variational Autoencoders (EBVAE), which is a general framework including popular VAE methods as special cases. Despite the widespread use of VAE, its theoretical aspects are less explored in the literature. Motivated by this, we establish a general theoretical framework for analyzing the excess risk associated with EBVAE under the setting of density estimation, covering both parametric and nonparametric cases, through the lens of M-estimation. As an application, we analyze the excess risk of the commonly-used EBVAE with Gaussian models and highlight the importance of covariance matrices of Gaussian encoders and decoders in obtaining a good statistical guarantee, shedding light on the empirical observations reported in the literature. 

   more » « less
2. Bayesian Pyramids: identifiable multilayer discrete latent structure models for discrete data

   https://doi.org/10.1093/jrsssb/qkad010  

   Gu, Yuqi; Dunson, David B (March 2023, Journal of the Royal Statistical Society Series B: Statistical Methodology)

   Abstract High-dimensional categorical data are routinely collected in biomedical and social sciences. It is of great importance to build interpretable parsimonious models that perform dimension reduction and uncover meaningful latent structures from such discrete data. Identifiability is a fundamental requirement for valid modeling and inference in such scenarios, yet is challenging to address when there are complex latent structures. In this article, we propose a class of identifiable multilayer (potentially deep) discrete latent structure models for discrete data, termed Bayesian Pyramids. We establish the identifiability of Bayesian Pyramids by developing novel transparent conditions on the pyramid-shaped deep latent directed graph. The proposed identifiability conditions can ensure Bayesian posterior consistency under suitable priors. As an illustration, we consider the two-latent-layer model and propose a Bayesian shrinkage estimation approach. Simulation results for this model corroborate the identifiability and estimatability of model parameters. Applications of the methodology to DNA nucleotide sequence data uncover useful discrete latent features that are highly predictive of sequence types. The proposed framework provides a recipe for interpretable unsupervised learning of discrete data and can be a useful alternative to popular machine learning methods. 

   more » « less
3. Empirical Bayes PCA in High Dimensions

   https://doi.org/10.1111/rssb.12490  

   Zhong, Xinyi; Su, Chang; Fan, Zhou (January 2022, Journal of the Royal Statistical Society Series B: Statistical Methodology)

   Abstract When the dimension of data is comparable to or larger than the number of data samples, principal components analysis (PCA) may exhibit problematic high-dimensional noise. In this work, we propose an empirical Bayes PCA method that reduces this noise by estimating a joint prior distribution for the principal components. EB-PCA is based on the classical Kiefer–Wolfowitz non-parametric maximum likelihood estimator for empirical Bayes estimation, distributional results derived from random matrix theory for the sample PCs and iterative refinement using an approximate message passing (AMP) algorithm. In theoretical ‘spiked’ models, EB-PCA achieves Bayes-optimal estimation accuracy in the same settings as an oracle Bayes AMP procedure that knows the true priors. Empirically, EB-PCA significantly improves over PCA when there is strong prior structure, both in simulation and on quantitative benchmarks constructed from the 1000 Genomes Project and the International HapMap Project. An illustration is presented for analysis of gene expression data obtained by single-cell RNA-seq. 

   more » « less
4. Numerical Approaches to Division and Label Structured Population Models

   Santiago, Fabian; Flores, Kevin B.; Sindi, Suzanne S. (October 2020, Letters in biomathematics)

   null (Ed.)

   Division and label structured population models (DLSPMs) are a class of partial differential equations (PDEs) that have been used to study intracellular dynamics in dividing cells. DLSPMs have improved the understanding of cell proliferation assays involving measurements such as fluorescent label decay, protein production, and prion aggregate amplification. One limitation in using DLSPMs is the significant computational time required for numerical approximations, especially for models with complex biologically relevant dynamics. Here we develop a novel numerical and theoretical framework involving a recursive formulation for a class of DLSPMs. We develop this framework for a population of dividing cells with an arbitrary functional form describing the intracellular dynamics. We found that, compared to previous methods, our framework is faster and more accurate. We illustrate our approach on three common models for intracellular dynamics and discuss the potential impact of our findings in the context of data-driven methods for parameter estimation. 

   more » « less
5. The Bayesian Prophet: A Low-Regret Framework for Online Decision Making

   https://doi.org/10.1287/mnsc.2020.3624  

   Vera, Alberto; Banerjee, Siddhartha (March 2021, Management Science)

   null (Ed.)

   We develop a new framework for designing online policies given access to an oracle providing statistical information about an off-line benchmark. Having access to such prediction oracles enables simple and natural Bayesian selection policies and raises the question as to how these policies perform in different settings. Our work makes two important contributions toward this question: First, we develop a general technique we call compensated coupling, which can be used to derive bounds on the expected regret (i.e., additive loss with respect to a benchmark) for any online policy and off-line benchmark. Second, using this technique, we show that a natural greedy policy, which we call the Bayes selector, has constant expected regret (i.e., independent of the number of arrivals and resource levels) for a large class of problems we refer to as “online allocation with finite types,” which includes widely studied online packing and online matching problems. Our results generalize and simplify several existing results for online packing and online matching and suggest a promising pathway for obtaining oracle-driven policies for other online decision-making settings. This paper was accepted by George Shanthikumar, big data analytics. 

   more » « less