Model and training specifications:
We use MCAN [1]}, a state-of-the-art Transformer-like model for VQA. Self-attention layers for question features and visual features, as well as the question-guided attention layers of visual features, are stacked one over another to build a deep model. Dropout is applied in every attention layer (after the softmax and before residual layer [2]}) and fully-connected layer to prevent overfitting [1]}, resulting in 62 dropout layers in total. Experiments are conducted using the code of [1]} as basis. Detailed experiment setting is in Appendix REF .
<FIGURE><TABLE>