Hate speech detection With the advance of deep learning, hate speech detection studies have utilized neural networks [1]}, [2]}, and word embedding methods [3]}. More recently, Transformer-based [4]} models have shown remarkable results. In hate speech detection, BERT has been adopted for various studies as hate speech detection can be considered a classification task. [5]} and [6]} compared a BERT-based model with Recurrent Neural Networks (RNNs)-based models and showed the BERT-based model outperforms other models. Furthermore, some studies have considered the model's bias and explainability. [7]} improved accuracy and reduced unintended bias by adopting multi-task learning that predicts toxicity of text and target group labels as additional information. [8]} utilized rationales of the dataset as additional information for finetuning BERT to deal with the bias and explainability. To improve performance in terms of the two considerations, we propose a more effective finetuning approach based on BERT and the rationales by adopting the pre-training framework.
