@inproceedings{
zhang2024unifl,
title={Uni{FL}: Improve Latent Diffusion Model via Unified Feedback Learning},
author={Jiacheng Zhang and Jie Wu and Yuxi Ren and Xin Xia and Huafeng Kuang and Pan Xie and Jiashi Li and Xuefeng Xiao and Weilin Huang and Shilei Wen and Lean Fu and Guanbin Li},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=sy2SmstDOB}
}
Latent diffusion models (LDM) have revolutionized text-to-image generation, leading to the proliferation of various advanced models and diverse downstream applications. However, despite these significant advancements, current diffusion models still suffer from several limitations, including inferior visual quality, inadequate aesthetic appeal, and inefficient inference, without a comprehensive solution in sight. To address these challenges, we present **UniFL**, a unified framework that leverages feedback learning to enhance diffusion models comprehensively. UniFL stands out as a universal, effective, and generalizable solution applicable to various diffusion models, such as SD1.5 and SDXL.
Notably, UniFL consists of three key components: perceptual feedback learning, which enhances visual quality; decoupled feedback learning, which improves aesthetic appeal; and adversarial feedback learning, which accelerates inference.
In-depth experiments and extensive user studies validate the superior performance of our method in enhancing generation quality and inference acceleration. For instance, UniFL surpasses ImageReward by 17\% user preference in terms of generation quality and outperforms LCM and SDXL Turbo by 57\% and 20\% general preference with 4-step inference.
UniFL: Improve Latent Diffusion Model via Unified Feedback Learning
Jiacheng Zhang Jie Wu Yuxi Ren Xin Xia Huafeng Kuang Pan Xie Jiashi Li Xuefeng Xiao Weilin Huang Shilei Wen Lean Fu Guanbin Li