@inproceedings{
chakrabarty2024lomoe,
title={Lo{MOE}: Localized Multi-Object Editing via Multi-Diffusion},
author={Goirik Chakrabarty and Aditya Chandrasekar and Ramya Hebbalaguppe and Prathosh AP},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=v4RnEVediE}
}
Recent developments in diffusion models have demonstrated an exceptional capacity to generate high-quality prompt-conditioned image edits. Nevertheless, previous approaches have primarily relied on textual prompts for image editing, which tend to be less effective when making precise edits to specific objects or fine-grained regions within a scene containing single/multiple objects. We introduce a novel framework for zero-shot localized multi-object editing through a multi-diffusion process to overcome this challenge. This framework empowers users to perform various operations on objects within an image, such as adding, replacing, or editing $\textbf{many}$ objects in a complex scene $\textbf{in one pass}$. Our approach leverages foreground masks and corresponding simple text prompts that exert localized influences on the target regions resulting in high-fidelity image editing. A combination of cross-attention and background preservation losses within the latent space ensures that the characteristics of the object being edited are preserved while simultaneously achieving a high-quality, seamless reconstruction of the background with fewer artifacts compared to the state-of-the-art (SOTA). We also curate and release a dataset dedicated to multi-object editing, named $\texttt{LoMOE}$-Bench. Our experiments against existing SOTA demonstrate the improved effectiveness of our approach in terms of both image editing quality, and inference speed.
LoMOE: Localized Multi-Object Editing via Multi-Diffusion
Goirik Chakrabarty Aditya Chandrasekar Ramya Hebbalaguppe Prathosh AP