@inproceedings{
wang2024observe,
title={Observe before Generate: Emotion-Cause aware Video Caption for Multimodal Emotion Cause Generation in Conversations},
author={Fanfan Wang and Heqing Ma and Xiangqing Shen and Jianfei Yu and Rui Xia},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=Pq63G43jkQ}
}
Emotion cause analysis has attracted increasing attention in recent years.
Extensive research has been dedicated to multimodal emotion recognition in conversations.
However, the integration of multimodal information with emotion cause remains underexplored.
Existing studies merely extract utterances or spans from conversations as cause evidence, which may not be concise and clear enough, especially the lack of explicit descriptions of other modalities, making it difficult to intuitively understand the causes. 
To address these limitations, we introduce a new task named Multimodal Emotion Cause Generation in Conversations (MECGC), which aims to generate an abstractive summary describing the causes that trigger the given emotion based on the multimodal context of conversations. 
We accordingly construct a dataset named ECGF that contains 1,374 conversations and 7,690 emotion instances from TV series.
We further develop a generative framework that first generates emotion-cause aware video captions (Observe) and then facilitates the generation of emotion causes (Generate).
The captioning model is trained with examples synthesized by a Multimodal Large Language Model (MLLM). 
Experimental results demonstrate the effectiveness of our framework and the significance of multimodal information for emotion cause analysis.
Observe before Generate: Emotion-Cause aware Video Caption for Multimodal Emotion Cause Generation in Conversations
Fanfan Wang Heqing Ma Xiangqing Shen Jianfei Yu Rui Xia