Neural Inverse Transform Sampler
Any explicit functional representation $f$ of a density is hampered by two
main obstacles when we wish to use it as a generative model: designing $f$ so
that sampling is fast, and estimating $Z = \int f$ so that $Z^{-1}f$ integrates
to 1. This becomes increasingly complicated as $f$ itself becomes complicated.
In this paper, we show that when modeling one-dimensional conditional densities
with a neural network, $Z$ can be exactly and efficiently computed by letting
the network represent the cumulative distribution function of a target density,
and applying a generalized fundamental theorem of calculus. We also derive a
fast algorithm for sampling from the resulting representation by the inverse
transform method. By extending these principles to higher dimensions, we
introduce the \textbf{Neural Inverse Transform Sampler (NITS)}, a novel deep
learning framework for modeling and sampling from general, multidimensional,
compactly-supported probability densities. NITS is a highly expressive density
estimator that boasts end-to-end differentiability, fast sampling, and exact
and cheap likelihood evaluation. We demonstrate the applicability of NITS by
applying it to realistic, high-dimensional density estimation tasks:
likelihood-based generative modeling on the CIFAR-10 dataset, and density
estimation on the UCI suite of benchmark datasets, where NITS produces
compelling results rivaling or surpassing the state of the art.