There have been significant studies focusing on improving word embeddings [16, 17] , phrase embeddings [18] , sentence embeddings <cite>[8</cite>, 19] , language models [20] through multi-modal learning of vision and language. Among all studies, <cite>[8]</cite> is the first to apply skip-gram-like intuition (predicting multiple modalities from langauge) to joint learning of language and vision in the perspective of general sentence representations.