@inproceedings{wu-etal-2024-interpretable,
    title = "Interpretable Short Video Rumor Detection Based on Modality Tampering",
    author = "Wu, Kaixuan  and
      Lin, Yanghao  and
      Cao, Donglin  and
      Lin, Dazhen",
    editor = "Calzolari, Nicoletta  and
      Kan, Min-Yen  and
      Hoste, Veronique  and
      Lenci, Alessandro  and
      Sakti, Sakriani  and
      Xue, Nianwen",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.804",
    pages = "9180--9189",
    abstract = "With the rapid development of social media and short video applications in recent years, browsing short videos has become the norm. Due to its large user base and unique appeal, spreading rumors via short videos has become a severe social problem. Many methods simply fuse multimodal features for rumor detection, which lack interpretability. For short video rumors, rumor makers create rumors by modifying and/or splicing different modal information, so we should consider how to detect rumors from the perspective of modality tampering. Inspired by cross-modal contrastive learning, we propose a novel short video rumor detection framework by designing two pretraining tasks: modality tampering detection and inter-modal matching, imbuing the model with the ability to detect modality tampering and employing it for downstream rumor detection tasks. In addition, we design an interpretability mechanism to make the rumor detection results more reasonable by backtracking the model{'}s decision-making process. The experimental results show that the method on the short video rumor dataset has an improvement of about 4.6{\%}-12{\%} in macro-F1 compared with other models and can explain whether the short video is a rumor or not through the perspective of modality tampering.",
}

With the rapid development of social media and short video applications in recent years, browsing short videos has become the norm. Due to its large user base and unique appeal, spreading rumors via short videos has become a severe social problem. Many methods simply fuse multimodal features for rumor detection, which lack interpretability. For short video rumors, rumor makers create rumors by modifying and/or splicing different modal information, so we should consider how to detect rumors from the perspective of modality tampering. Inspired by cross-modal contrastive learning, we propose a novel short video rumor detection framework by designing two pretraining tasks: modality tampering detection and inter-modal matching, imbuing the model with the ability to detect modality tampering and employing it for downstream rumor detection tasks. In addition, we design an interpretability mechanism to make the rumor detection results more reasonable by backtracking the model{'}s decision-making process. The experimental results show that the method on the short video rumor dataset has an improvement of about 4.6{\%}-12{\%} in macro-F1 compared with other models and can explain whether the short video is a rumor or not through the perspective of modality tampering.
Wu, Kaixuan Lin, Yanghao Cao, Donglin Lin, Dazhen
Interpretable Short Video Rumor Detection Based on Modality Tampering