The core task of VLP is to model the interactions between images and texts and there are two types of architectures for this: 1) The single-stream models like ImageBERT [1]}, Unicoder-VL [2]} concatenate patch-wise or regional visual and textual embeddings and feed them to one encoder. 2) The dual-stream models like ViLBERT [3]} and CLIP [4]} obtain visual and textual embeddings with separate encoders.
