
The main motivation behind this methodology is the recent success that transformer models had in various NLP tasks [1]} including offensive language identification [2]}, [3]}, [4]}. Most of these transformer-based approaches take the final hidden state of the first token ([CLS]) from the transformer as the representation of the whole sequence and a simple softmax classifier is added to the top of the transformer model to predict the probability of a class label [5]}. However, as previously mentioned, these models classify whole comments or documents and do not identify the spans that make a text offensive. Since the objective of this task is to identify offensive spans rather than classifying the whole comment, we followed a different architecture.
