CLIP [1]} is a powerful vision-language pre-training model, which shows surprisingly strong performance in associating the visual and textual concepts. CLIP is a two-stream method: it contains a vision encoder \({\cal E}_{\text{image}}\)  and a text encoder \({\cal E}_{\text{text}}\) . For any given image-text paired data \(\lbrace {\cal I}, {\cal T}\rbrace \)  , their semantic similarity can be estimated by computing the cosine distance between \({\cal E}_{\text{image}}({\cal I})\)  and \({\cal E}_{\text{text}}({\cal T})\) .
