Towards generalizability, we extend Harm-P by redesigning the existing data splits as shown in Table REF . We call the resulting dataset Ext-Harm-P. It contains a total of 4,446 examples including 1,594 harmful and 2,852 non-harmful; both categories have references to a number of entities.
For training, we use the harmful memes provided as part of the original dataset [1]}, which we re-annotate for the fine-grained entities that are being targeted harmfully as positive samples (harmful targets). This is matched with twice as many negative samples (not-harmful targets). For negative targets, we use the top-2 entities from the original entity lexicon, which are not labeled for harmfulness and have the highest lexical similarity with the meme text [2]}. This at least ensures lexical similarity with the entities referenced within a meme, thereby facilitating a confounding effect [3]} as well. For the test set, all the entities are first extracted automatically using named entity recognition (NER) and person identification (PID)NER using SpaCy & PID using http://github.com/ageitgey/face_recognition.. This is followed by manual annotation of the test set.
<FIGURE><FIGURE>