@inproceedings{
fu2024fedcafe,
title={Fed{CAFE}: Federated Cross-Modal Hashing with Adaptive Feature Enhancement},
author={Ting Fu and Yu-Wei Zhan and Chong-Yu Zhang and Xin Luo and Zhen-Duo Chen and Yongxin Wang and Xun Yang and Xin-Shun Xu},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=54MrV3qXmL}
}
Deep Cross-Modal Hashing (CMH) has become one of the most popular solutions for cross-modal retrieval. Existing methods need to first collect data and then be trained with these accumulated data. However, in real world, data may be generated and possessed by different owners. Considering the concerns about privacy, data may not be shared or transmitted, leading to the failure of sufficient training of CMH. To solve the problem, we propose a new framework called Federated Cross-modal Hashing with Adaptive Feature Enhancement (FedCAFE). FedCAFE is a federated method which could use distributed data to train existing CMH methods under the privacy protection. To overcome the data heterogeneity challenge of distributed data and improve the generalization ability of global model, FedCAFE is endowed with a novel adaptive feature enhancement module and a new weighted aggregation strategy. Besides, it could fully utilize the rich global information carried in the global model to constrain the model during the local training process. We have conducted extensive experiments on four widely-used datasets in CMH domain with both IID and non-IID settings. The reported results demonstrate that the proposed FedCAFE achieves better performance than several state-of-the-art baselines. As the topic that training deep CMH in federated scenario is in its infancy, we plan to release the code and data to boost the development of the field. However, considering restriction of anonymous submission and size limitation, we could only upload the source code of FedCAFE as supplementary materials for peer review at the present stage.
FedCAFE: Federated Cross-Modal Hashing with Adaptive Feature Enhancement
Ting Fu Yu-Wei Zhan Chong-Yu Zhang Xin Luo Zhen-Duo Chen Yongxin Wang Xun Yang Xin-Shun Xu