@inproceedings{
chen2024simplifying,
title={Simplifying Cross-modal Interaction via Modality-Shared Features for {RGBT} Tracking},
author={LiQiu Chen and Yuqing Huang and Hengyu li and Zikun Zhou and Zhenyu He},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=oDEqOhKYoO}
}
Thermal infrared (TIR) data exhibits higher tolerance to extreme environments, making it a valuable complement to RGB data in tracking tasks. RGB-T tracking aims to leverage information from both RGB and TIR images for stable and robust tracking. However, existing RGB-T tracking methods often face challenges due to significant modality differences and selective emphasis on interactive information, leading to inefficiencies in the cross-modal interaction process. To address these issues, we propose a novel Integrating Interaction into Modality-shared Fearues with ViT(IIMF) framework, which is a simplified cross-modal interaction network including modality-shared, RGB modality-specific, and TIR modality-specific branches. Modality-shared branch aggregates modality-shared information and implements inter-modal interaction with the Vision Transformer(ViT). Specifically, our approach first extracts modality-shared features from RGB and TIR features using a cross-attention mechanism. Furthermore, we design a Cross-Attention-based Modality-shared Information Aggregation (CAMIA) module to further aggregate modality-shared information with modality-shared tokens.
Simplifying Cross-modal Interaction via Modality-Shared Features for RGBT Tracking
LiQiu Chen Yuqing Huang Hengyu li Zikun Zhou Zhenyu He