We make use of 3 referring expression datasets in our work, all collected on top of the Microsoft COCO image collection [24] . One dataset, RefCOCOg <cite>[26]</cite> is collected in a non-interactive setting, while the other two datasets, RefCOCO and RefCOCO+, are collected interactively in a two-player game [19] .