[leftmargin=*]
–
Unimodal Systems: \(\blacktriangleright \)  VGG16, VIT: For the unimodal (image-only) systems, we use two well-known models: VGG16 [1]} and VIT (Vision Transformers) that emulate a Transformer-based application jointly over textual tokens and image patches [2]}.
\(\blacktriangleright \)  GRU, XLNet: For the unimodal (text-only) systems, we use GRU [3]}, which adaptively captures temporal dependencies, and XLNet [4]}, which implements a generalized auto-regressive pre-training strategy.
–
Multimodal Systems: \(\blacktriangleright \)  MMF Transformer: This is a multimodal Transformer model that uses visual and language tokens with self-attention.http://mmf.sh/docs/notes/model_zoo
\(\blacktriangleright \)  MMBT: Multimodal Bitransformer [5]} captures the intra-modal and the inter-modal dynamics.
\(\blacktriangleright \)  ViLBERT CC: Vision and Language BERT [6]}, pre-trained on CC [7]}, is a strong model with task-agnostic joint representation.
\(\blacktriangleright \)  Visual BERT COCO: Visual BERT [8]}, pre-trained on the MS COCO dataset [9]}.

<TABLE>