@inproceedings{
kim2024synctweedies,
title={SyncTweedies: A General Generative Framework Based on Synchronized Diffusions},
author={Jaihoon Kim and Juil Koo and Kyeongmin Yeo and Minhyuk Sung},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=06Vt6f2js7}
}
We introduce a general diffusion synchronization framework for generating diverse visual content, including ambiguous images, panorama images, 3D mesh textures, and 3D Gaussian splats textures, using a pretrained image diffusion model. We first present an analysis of various scenarios for synchronizing multiple diffusion processes through a canonical space. Based on the analysis, we introduce a synchronized diffusion method, SyncTweedies, which averages the outputs of Tweedie’s formula while conducting denoising in multiple instance spaces. Compared to previous work that achieves synchronization through finetuning, SyncTweedies is a zero-shot method that does not require any finetuning, preserving the rich prior of diffusion models trained on Internet-scale image datasets without overfitting to specific domains. We verify that SyncTweedies offers the broadest applicability to diverse applications and superior performance compared to the previous state-of-the-art for each application. Our project page is at https://synctweedies.github.io.
SyncTweedies: A General Generative Framework Based on Synchronized Diffusions
Jaihoon Kim Juil Koo Kyeongmin Yeo Minhyuk Sung