@inproceedings{
zhu2024inmunet,
title={InMu-Net: Advancing Multi-modal Intent Detection via Information Bottleneck and Multi-sensory Processing},
author={Zhihong Zhu and Xuxin Cheng and Zhaorun Chen and Yuyan Chen and Yunyan Zhang and Xian Wu and Yefeng Zheng and Bowen Xing},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=CuX8NF7XJq}
}
Multi-modal intent detection (MID) aims to comprehend users' intentions through diverse modalities, which has received widespread attention in dialogue systems. Despite the promising advancements in complex fusion mechanisms or architecture designs, challenges remain due to: (1) various noise and redundancy in both visual and audio modalities and (2) long-tailed distributions of intent categories. In this paper, to tackle the above two issues, we propose InMu-Net, a simple yet effective framework for MID from the $\textbf{In}$formation bottleneck and $\textbf{Mu}$lti-sensory processing perspective. Our contributions lie in three aspects. First, we devise a $\textit{denoising bottleneck module}$ to filter out the intent-irrelevant information in the fused feature; Second, we introduce a $\textit{saliency preservation loss}$ to prevent the dropping of intent-relevant information; Ultimately, $\textit{kurtosis regulation}$ is introduced to maintain representation smoothness during the filtering process, mitigating the adverse impact of the long tail distribution. Comprehensive experiments on two MID benchmark datasets demonstrate the effectiveness of InMu-Net and its vital components. Impressively, a series of analyses reveal our denoising potential and robustness in low-resource, modality corruption, cross-architecture and cross-task scenarios.
InMu-Net: Advancing Multi-modal Intent Detection via Information Bottleneck and Multi-sensory Processing
Zhihong Zhu Xuxin Cheng Zhaorun Chen Yuyan Chen Yunyan Zhang Xian Wu Yefeng Zheng Bowen Xing