We evaluate on image prompts generated by each image encoder on multiple image captioning datasets: MSCOCO [1]} and NoCaps [2]}, as well as the VQA2 [3]} visual question-answering dataset. Following convention from SimVLM and MAGMA, we input the prefix “A picture of" after every image to prompt the model. Like in previous work, we find that this is a favorable prompt which tends to increase performance.
