@inproceedings{
zhou2024foreground,
title={Foreground Harmonization and Shadow Generation for  Composite Image},
author={Jing Zhou and Ziqi Yu and Zhongyun Bao and Gang Fu and Weilei He and Chao Liang and Chunxia Xiao},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=FxcoztYv98}
}
We propose a method for light and shadow editing of outdoor disharmonious composite images, including foreground harmonization and cast shadow generation. Most existing work can only perform foreground appearance editing tasks or only focus on shadow generation. In fact, lighting not only affects the brightness and color of objects, but also produces corresponding cast shadows. In recent years, diffusion models have demonstrated their strong generative capabilities, and due to their iterative denoising properties, they have a significant advantage in image restoration tasks. But it fails to preserve content structure of image. In this purpose, we propose an effective model to tackle the problem of foreground light-shadow editing. Specifically, we use a coarse shadow prediction module (SP) to generate coarse shadows for foreground objects. Then, we use the predicted results as prior knowledge to guide the generation of harmony diffusion model. In this process, the primary task is to learn lighting variation to harmonize foreground regions. The secondary task is to generate high-quality cast shadow containing more details. Considering that existing datasets do not support the dual tasks of image harmonization and shadow generation, we construct a real outdoor dataset, IH-SG, covering various lighting conditions. Extensive experiments conducted on existing benchmark datasets and the IH-SG dataset demonstrate the superiority of our method.
Foreground Harmonization and Shadow Generation for  Composite Image
Jing Zhou Ziqi Yu Zhongyun Bao Gang Fu Weilei He Chao Liang Chunxia Xiao