We use our model as an image encoder and extract the visual features for all images in the HatefulMemes dataset.
For all models, we extract the features before the final pooling layer in order to preserve the spatial information.
We use BERT-Base [1]} as the text encoder.
We concatenate the image features with the BERT text features and train an MLP head on top.
We use the AdamW optimizer [2]} with epsilon \(1e-8\) , a learning rate of \(8e-5\) .
We use a linear learning rate warmup for \(2,000\)  iterations followed by step decays value by \(1e-5\)  every 500 iterations.
We train We use open source library https://github.com/facebookresearch/mmf. for a total of \(22,000\)  iterations with a batch size of 64.
We report the best ROC AUC metric on the dev set during training.
