@inproceedings{
pang2024attndreambooth,
title={AttnDreamBooth: Towards Text-Aligned Personalized Text-to-Image Generation},
author={Lianyu Pang and Jian Yin and Baoquan Zhao and Feize Wu and Fu Lee Wang and Qing Li and Xudong Mao},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=4bINoegDcm}
}
Recent advances in text-to-image models have enabled high-quality personalized image synthesis based on user-provided concepts with flexible textual control. In this work, we analyze the limitations of two primary techniques in text-to-image personalization: Textual Inversion and DreamBooth. When integrating the learned concept into new prompts, Textual Inversion tends to overfit the concept, while DreamBooth often overlooks it. We attribute these issues to the incorrect learning of the embedding alignment for the concept. To address this, we introduce AttnDreamBooth, a novel approach that separately learns the embedding alignment, the attention map, and the subject identity across different training stages. We also introduce a cross-attention map regularization term to enhance the learning of the attention map. Our method demonstrates significant improvements in identity preservation and text alignment compared to the baseline methods.
AttnDreamBooth: Towards Text-Aligned Personalized Text-to-Image Generation
Lianyu Pang Jian Yin Baoquan Zhao Feize Wu Fu Lee Wang Qing Li Xudong Mao