To more effectively describe the relevance between the image and text, several works [1]} rely on local features. To be specific, they represent an image by a set of bounding box features which are extracted by pre-trained object detectors. The detected bounding boxes are the candidate locations of objects in the image. Meanwhile, they represent the sentence by a set of word features. Then the relevance between the image and the text sentence is obtained by set-to-set matching between word features and bounding box features. When conducting the set-to-set matching, the relevant word-box pairs with a large similarity score can be easily identified.
