Our unimodal models are based on RoBERTa-Large [1]} and DeIT [2]} for text and image, respectively, and the overall structure is shown in Fig. REF . If there is no multimodal pretraining for these unimodal models, it is difficult to leverage the pretrained unimodal models because the representations of other modals are not mutually understood. In other words, RoBERTa does not understand the image representation because it is learned only with text data. We show the effect of pretraining through experiments in Section . In addition, since the output of the pretrained multimodal DeIT can be regarded as the output of RoBERTa for the object description, it can be used in various ways. Details are explained in Section REF .
