SPI-GAN: Distilling Score-based Generative Models with Straight-Path Interpolations
Score-based generative models (SGMs) are a recently proposed paradigm for
deep generative tasks and now show the state-of-the-art sampling performance.
It is known that the original SGM design solves the two problems of the
generative trilemma: i) sampling quality, and ii) sampling diversity. However,
the last problem of the trilemma was not solved, i.e., their training/sampling
complexity is notoriously high. To this end, distilling SGMs into simpler
models, e.g., generative adversarial networks (GANs), is gathering much
attention currently. We present an enhanced distillation method, called
straight-path interpolation GAN (SPI-GAN), which can be compared to the
state-of-the-art shortcut-based distillation method, called denoising diffusion
GAN (DD-GAN). However, our method corresponds to an extreme method that does
not use any intermediate shortcut information of the reverse SDE path, in which
case DD-GAN fails to obtain good results. Nevertheless, our straight-path
interpolation method greatly stabilizes the overall training process. As a
result, SPI-GAN is one of the best models in terms of the sampling
quality/diversity/time for CIFAR-10, CelebA-HQ-256, and LSUN-Church-256.