@inproceedings{
hu2024cracking,
title={Cracking the Code of Juxtaposition: Can {AI} Models Understand the Humorous Contradictions},
author={Zhe Hu and Tuo Liang and Jing Li and Yiren Lu and Yunlai Zhou and Yiran Qiao and Jing Ma and Yu Yin},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=bCMpdaQCNW}
}
Recent advancements in large vision language models have demonstrated remarkable proficiency across a wide range of tasks. 
Yet, these models still struggle with understanding the nuances of human humor through juxtaposition, particularly when it involves nonlinear narratives that underpin many jokes and humor cues.  This paper investigates this challenge by focusing on comics with contradictory narratives, where each comic consists of two panels that create a humorous contradiction. We introduce the YesBut benchmark, which comprises tasks of varying difficulty aimed at assessing AI's capabilities in recognizing and interpreting these comics, ranging from literal content comprehension to deep narrative reasoning. Through extensive experimentation and analysis of recent commercial or open-sourced large vision language models, we assess their capability to comprehend the complex interplay of the narrative humor inherent in these comics. Our results show that even the state-of-the-art models still struggle with this task. Our findings offer insights into the current limitations and potential improvements for AI in understanding human creative expressions.
Cracking the Code of Juxtaposition: Can AI Models Understand the Humorous Contradictions
Zhe Hu Tuo Liang Jing Li Yiren Lu Yunlai Zhou Yiran Qiao Jing Ma Yu Yin