@inproceedings{
su2024ibmea,
title={{IBMEA}: Exploring Variational Information Bottleneck for Multi-modal Entity Alignment},
author={Taoyu Su and Jiawei Sheng and Shicheng Wang and Xinghua Zhang and Hongbo Xu and Tingwen Liu},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=bFXYw5fG8t}
}
Multi-modal entity alignment (MMEA) aims to identify equivalent entities between multi-modal knowledge graphs (MMKGs), where entities can be associated with related images.
Most existing studies rely heavily on the automatically learned multi-modal fusion modules, which may allow redundant information such as misleading clues in the generated entity representations, impeding the feature consistency of equivalent entities.
To this end, we propose a variational framework for MMEA via information bottleneck, termed as IBMEA, by emphasizing alignment-relevant information while suppressing alignment-irrelevant information in entity representations.
Specifically, we first develop multi-modal variational encoders that represent modal-specific features as probability distributions.
Then, we propose four modal-specific information bottleneck regularizers to limit the misleading clues in the modal-specific entity representations. 
Finally, we propose a modal-hybrid information contrastive regularizer to integrate modal-specific representations and ensure the similarity of equivalent entities between MMKGs to achieve MMEA. 
We conduct extensive experiments on 2 cross-KG and 3 bilingual MMEA datasets.
Experimental results demonstrate that our model consistently outperforms previous state-of-the-art methods, and also shows promising and robust performance especially in the low-resource and high-noise data scenarios.
IBMEA: Exploring Variational Information Bottleneck for Multi-modal Entity Alignment
Taoyu Su Jiawei Sheng Shicheng Wang Xinghua Zhang Hongbo Xu Tingwen Liu