- The motivation behind the use of multilingual models comes from recent work [1]} which used transfer learning and cross lingual embeddings. This study showed that cross-lingual transformers like XLM-R [2]} can be trained on an English dataset and have the model weights saved to detect offensive language in other languages outperforming monolingual models trained on the target language dataset. We used a similar methodology but for the token classification architecture instead. We used XLM-R cross-lingual transformer model [2]} as the Transformer in Figure REF  on TSDTrain and carried out evaluations on the Danish and Greek datasets. We release two multilingual models; multilingual-base based on XLM-R base model and multilingual-large based on XLM-R large model.
