Flowification: Everything is a Normalizing Flow
We develop a method that can be used to turn any multi-layer perceptron or
convolutional network into a normalizing flow. In some cases this requires the
addition of uncorrelated noise to the model but in the simplest case no
additional parameters. The technique we develop can be applied to a broad range
of architectures, allowing them to be used for a wide range of tasks. Our
models also allow existing density estimation techniques to be combined with
high performance feature extractors. In contrast to standard density estimation
techniques that require specific architectures and specialized knowledge, our
approach can leverage design knowledge from other domains and is a step closer
to the realization of general purpose architectures. We investigate the
efficacy of linear and convolutional layers for the task of density estimation
on standard datasets. Our results suggest standard layers lack something
fundamental in comparison to other normalizing flows.