Codec at SemEval-2022 Task 5: Multi-Modal Multi-Transformer Misogynous Meme Classification Framework
In this paper we describe our work towards building a generic framework for
both multi-modal embedding and multi-label binary classification tasks, while
participating in task 5 (Multimedia Automatic Misogyny Identification) of
SemEval 2022 competition.
  Since pretraining deep models from scratch is a resource and data hungry
task, our approach is based on three main strategies. We combine different
state-of-the-art architectures to capture a wide spectrum of semantic signals
from the multi-modal input. We employ a multi-task learning scheme to be able
to use multiple datasets from the same knowledge domain to help increase the
model's performance. We also use multiple objectives to regularize and fine
tune different system components.