A Multimodal Fusion Approach By Exploiting Concept Interactions for Efficient Multimedia Analysis
Multimedia data intrinsically contains multimodal information in it. In order to obtain a successful multimedia analysis, all available information should be utilized by following a multimodal approach. In addition, the interaction between concepts is another crucial source of information and helps to increase the fusion performance. The focus of this study is to show that fusing all available modalities along with the concept interactions can yield better results. The experiments conducted on TRECVID 2007 dataset validated the superiority of such combination over best single modality and alternative modality combinations. Combining all available modalities performs 10.5% relatively better than the best single modality in overall. Additionally, utilizing from other concept information provides a 5.6% relative performance gain over the multimodal fusion results.