Sparse Infinite Random Feature Latent Variable Modeling
We propose a non-linear, Bayesian non-parametric latent variable model where
the latent space is assumed to be sparse and infinite dimensional a priori
using an Indian buffet process prior. A posteriori, the number of instantiated
dimensions in the latent space is guaranteed to be finite. The purpose of
placing the Indian buffet process on the latent variables is to: 1.)
Automatically and probabilistically select the number of latent dimensions. 2.)
Impose sparsity in the latent space, where the Indian buffet process will
select which elements are exactly zero. Our proposed model allows for sparse,
non-linear latent variable modeling where the number of latent dimensions is
selected automatically. Inference is made tractable using the random Fourier
approximation and we can easily implement posterior inference through Markov
chain Monte Carlo sampling. This approach is amenable to many observation
models beyond the Gaussian setting. We demonstrate the utility of our method on
a variety of synthetic, biological and text datasets and show that we can
obtain superior test set performance compared to previous latent variable
models.