Recent work has shown that we can learn better visually grounded representations of sentences by training image-sentence ranking models on multiple languages (Gella et al., 2017; <cite>Kádár et al., 2018)</cite> . This line of research has focused on training models on datasets where the same images are annotated with sentences in multiple languages. In this paper, we consider the problem of training an image-sentence ranking model using imagecaption collections in different languages with nonoverlapping images drawn from different sources.