Fat-Tailed Variational Inference with Anisotropic Tail Adaptive Flows
While fat-tailed densities commonly arise as posterior and marginal
distributions in robust models and scale mixtures, they present challenges when
Gaussian-based variational inference fails to capture tail decay accurately. We
first improve previous theory on tails of Lipschitz flows by quantifying how
the tails affect the rate of tail decay and by expanding the theory to
non-Lipschitz polynomial flows. Then, we develop an alternative theory for
multivariate tail parameters which is sensitive to tail-anisotropy. In doing
so, we unveil a fundamental problem which plagues many existing flow-based
methods: they can only model tail-isotropic distributions (i.e., distributions
having the same tail parameter in every direction). To mitigate this and enable
modeling of tail-anisotropic targets, we propose anisotropic tail-adaptive
flows (ATAF). Experimental results on both synthetic and real-world targets
confirm that ATAF is competitive with prior work while also exhibiting
appropriate tail-anisotropy.