null
(Ed.)
Transformation-based methods have been an
attractive approach in non-parametric inference for problems such as unconditional and
conditional density estimation due to their
unique hierarchical structure that models the
data as flexible transformation of a set of
common latent variables. More recently,
transformation-based models have been used
in variational inference (VI) to construct flexible implicit families of variational distributions. However, their use in both nonparametric inference and variational inference lacks theoretical justification. We provide theoretical justification for the use of
non-linear latent variable models (NL-LVMs)
in non-parametric inference by showing that
the support of the transformation induced
prior in the space of densities is sufficiently
large in the L1 sense. We also show that,
when a Gaussian process (GP) prior is placed
on the transformation function, the posterior concentrates at the optimal rate up to
a logarithmic factor. Adopting the flexibility demonstrated in the non-parametric setting, we use the NL-LVM to construct an
implicit family of variational distributions,
deemed GP-IVI. We delineate sufficient conditions under which GP-IVI achieves optimal
risk bounds and approximates the true posterior in the sense of the Kullback–Leibler
divergence. To the best of our knowledge,
this is the first work on providing theoretical
guarantees for implicit variational inference.
more »
« less