Kantorovich Strikes Back! Wasserstein GANs are not Optimal Transport?
Wasserstein Generative Adversarial Networks (WGANs) are the popular
generative models built on the theory of Optimal Transport (OT) and the
Kantorovich duality. Despite the success of WGANs, it is still unclear how well
the underlying OT dual solvers approximate the OT cost (Wasserstein-1 distance,
$\mathbb{W}_{1}$) and the OT gradient needed to update the generator. In this
paper, we address these questions. We construct 1-Lipschitz functions and use
them to build ray monotone transport plans. This strategy yields pairs of
continuous benchmark distributions with the analytically known OT plan, OT cost
and OT gradient in high-dimensional spaces such as spaces of images. We
thoroughly evaluate popular WGAN dual form solvers (gradient penalty, spectral
normalization, entropic regularization, etc.) using these benchmark pairs. Even
though these solvers perform well in WGANs, none of them faithfully compute
$\mathbb{W}_{1}$ in high dimensions. Nevertheless, many provide a meaningful
approximation of the OT gradient. These observations suggest that these solvers
should not be treated as good estimators of $\mathbb{W}_{1}$, but to some
extent they indeed can be used in variational problems requiring the
minimization of $\mathbb{W}_{1}$.