Density Estimation with Autoregressive Bayesian Predictives
Bayesian methods are a popular choice for statistical inference in small-data
regimes due to the regularization effect induced by the prior, which serves to
counteract overfitting. In the context of density estimation, the standard
Bayesian approach is to target the posterior predictive. In general, direct
estimation of the posterior predictive is intractable and so methods typically
resort to approximating the posterior distribution as an intermediate step. The
recent development of recursive predictive copula updates, however, has made it
possible to perform tractable predictive density estimation without the need
for posterior approximation. Although these estimators are computationally
appealing, they tend to struggle on non-smooth data distributions. This is
largely due to the comparatively restrictive form of the likelihood models from
which the proposed copula updates were derived. To address this shortcoming, we
consider a Bayesian nonparametric model with an autoregressive likelihood
decomposition and Gaussian process prior, which yields a data-dependent
bandwidth parameter in the copula update. Further, we formulate a novel
parameterization of the bandwidth using an autoregressive neural network that
maps the data into a latent space, and is thus able to capture more complex
dependencies in the data. Our extensions increase the modelling capacity of
existing recursive Bayesian density estimators, achieving state-of-the-art
results on tabular data sets.