TITLE: VISA: An Ambiguous Subtitles Dataset for Visual Scene-Aware Machine Translation 

ABSTRACT: Existing multimodal machine translation (MMT) datasets consist of images and
video captions or general subtitles, which rarely contain linguistic ambiguity,
making visual information not so effective to generate appropriate
translations. We introduce VISA, a new dataset that consists of 40k
Japanese-English parallel sentence pairs and corresponding video clips with the
following key features: (1) the parallel sentences are subtitles from movies
and TV episodes; (2) the source subtitles are ambiguous, which means they have
multiple possible translations with different meanings; (3) we divide the
dataset into Polysemy and Omission according to the cause of ambiguity. We show
that VISA is challenging for the latest MMT system, and we hope that the
dataset can facilitate MMT research. The VISA dataset is available at:
https://github.com/ku-nlp/VISA.
Existing multimodal machine translation (MMT) datasets consist of images and
video captions or general subtitles, which rarely contain linguistic ambiguity,
making visual information not so effective to generate appropriate
translations. We introduce VISA, a new dataset that consists of 40k
Japanese-English parallel sentence pairs and corresponding video clips with the
following key features: (1) the parallel sentences are subtitles from movies
and TV episodes; (2) the source subtitles are ambiguous, which means they have
multiple possible translations with different meanings; (3) we divide the
dataset into Polysemy and Omission according to the cause of ambiguity. We show
that VISA is challenging for the latest MMT system, and we hope that the
dataset can facilitate MMT research. The VISA dataset is available at:
https://github.com/ku-nlp/VISA.
VISA: An Ambiguous Subtitles Dataset for Visual Scene-Aware Machine Translation
http://arxiv.org/abs/2201.08054v3