In the past years, we have witnessed rapid progress in both computer vision and NLP. Some deep learning models pretrained by large-scale dataset have achieved significantly better performance than traditional methods based on hand-crafted features. To be specific, convolutional neural network (CNN) [1]} has substantially improved the performance in image/video recognition. Meanwhile, its output of a CNN's hidden layer is an effective image/video representation which can be used for visual-to-visual retrieval. In parallel, the transformer [2]} and BERT [3]} has achieved substantial success in many NLP tasks. Similarly, the output of a BERT's hidden layer is also effective for text-to-text retrieval. Despite the CNN feature and BERT feature have achieved significant success in visual-to-visual and text-to-text retrieval tasks, the text-to-visual search in our task is still challenging.
<FIGURE>