Multimodal corpora such as Flickr30k [1] or MSCOCO [2] containing images along with natural language captions were made available for research. They were soon extended with speech modality: speech recordings for the captions of Flickr8k were collected by [3] via crowdsourcing; spoken captions for MSCOCO were generated using Google Text-To-Speech (TTS) by<cite> [4]</cite> and using Voxygen TTS by [5] ; extensions of these corpora to other languages than English, such as Japanese, were also introduced by [6] .