For the COCO dataset [1]}, we note that it introduces challenges from two perspectives, being both highly-structured, with each scene populated by many objects that hold intricate relations and dependencies, but also visually and semantically diverse, consisting of varied images from a wide range of domains. In order to isolate the former challenge of modeling compositional scenes from the latter important but different challenge of covering a diverse image distribution, we study training on a topical partition of COCO, named COCO\(_p\) , that groups images into 7 semantically-related splits, listed in table REF . To partition the dataset, we cluster t-SNE processed ResNet activations of the COCO images into 31 subsets, which are then semantically grouped into the 7 splits. We train the models on each split separately, and report the mean scores. As expected, and also empirically suggested by table REF , the partition leads to improved visual quality across all models – both baselines and new ones, likely due to the more uniform resulting training distributions.
