@inproceedings{
wang2024terf,
title={Te{RF}: Text-driven and Region-aware Flexible Visible and Infrared Image Fusion},
author={Hebaixu Wang and Hao Zhang and Xunpeng Yi and Xinyu Xiang and Leyuan Fang and Jiayi Ma},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=uvjZktmxMn}
}
The fusion of visible and infrared images aims to produce high-quality fusion images with rich textures and salient target information. Existing methods lack interactivity and flexibility in the execution of fusion. It is unfeasible to express the requirements to modify the fusion effect, and the different regions in the source images are treated equally across the identical fusion model, which causes the fusion homogenization and low distinction. Besides, their pre-defined fusion strategies invariably lead to monotonous effects, which are insufficiently comprehensive. They fail to adequately consider data credibility, scene illumination, and noise degradation inherent in the source information. To address these issues, we propose the Text-driven and Region-aware Flexible visible and infrared image fusion, termed as TeRF. On the one hand, we propose a flexible image fusion framework with multiple large language and vision models, which facilitates the visual-text interaction. On the other hand, we aggregate comprehensive fine-tuning paradigms for the different fusion requirements to build a unified fine-tuning pipeline. It allows the linguistic selection of the regions and effects, yielding visually appealing fusion outcomes. Extensive experiments demonstrate the competitiveness of our method both qualitatively and quantitatively compared to existing state-of-the-art methods.
TeRF: Text-driven and Region-aware Flexible Visible and Infrared Image Fusion
Hebaixu Wang Hao Zhang Xunpeng Yi Xinyu Xiang Leyuan Fang Jiayi Ma