In order to study the effects of incorporating self-attention mechanism in joint prediction of image and language features, we examine attention vectors for selected samples from MS-COCO dataset and compare them to associated images ( Figure 1 ). These findings show that visually grounding self-attended sentence representations helps to expose word-level visual features onto sentence representations <cite>[8]</cite> .