@inproceedings{
kim2024pagoda,
title={PaGo{DA}: Progressive Growing of a One-Step Generator from a Low-Resolution Diffusion Teacher},
author={Dongjun Kim and Chieh-Hsin Lai and Wei-Hsiang Liao and Yuhta Takida and Naoki Murata and Toshimitsu Uesaka and Yuki Mitsufuji and Stefano Ermon},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=h5zYGF68KH}
}
The diffusion model performs remarkable in generating high-dimensional content but is computationally intensive, especially during training. We propose Progressive Growing of Diffusion Autoencoder (PaGoDA), a novel pipeline that reduces the training costs through three stages: training diffusion on downsampled data, distilling the pretrained diffusion, and progressive super-resolution. With the proposed pipeline, PaGoDA achieves a $64\times$ reduced cost in training its diffusion model on $8\times$ downsampled data; while at the inference, with the single-step, it performs state-of-the-art on ImageNet across all resolutions from $64\times64$ to $512\times512$, and text-to-image. PaGoDA's pipeline can be applied directly in the latent space, adding compression alongside the pre-trained autoencoder in Latent Diffusion Models (e.g., Stable Diffusion). The code is available at https://github.com/sony/pagoda.
PaGoDA: Progressive Growing of a One-Step Generator from a Low-Resolution Diffusion Teacher
Dongjun Kim Chieh-Hsin Lai Wei-Hsiang Liao Yuhta Takida Naoki Murata Toshimitsu Uesaka Yuki Mitsufuji Stefano Ermon