Marginal Tail-Adaptive Normalizing Flows
Learning the tail behavior of a distribution is a notoriously difficult
problem. By definition, the number of samples from the tail is small, and deep
generative models, such as normalizing flows, tend to concentrate on learning
the body of the distribution. In this paper, we focus on improving the ability
of normalizing flows to correctly capture the tail behavior and, thus, form
more accurate models. We prove that the marginal tailedness of an
autoregressive flow can be controlled via the tailedness of the marginals of
its base distribution. This theoretical insight leads us to a novel type of
flows based on flexible base distributions and data-driven linear layers. An
empirical analysis shows that the proposed method improves on the accuracy --
especially on the tails of the distribution -- and is able to generate
heavy-tailed data. We demonstrate its application on a weather and climate
example, in which capturing the tail behavior is essential.