During the first workshop [1]}, an approach of ensembling 11 different machine learning and Neural Network models such as LR, SVM, RF, LSTM, etc., and transformers (RoBERTa, BERT, ALBERT, IndicBERT) gave averaged-weighted F1-scores of 0.93 for English comments. Similar performances were reported by a methodology consisting combination of contextualized string embedding, stacked word embeddings, and pooled document embedding with Recurrent Neural Network (RNN) [2]}. Several other participants, explored transformers individually such as RoBERTa [3]}, XML-R [4]}, XLM-RoBERTa [5]}, XLM-RoBERTa with TF-IDF [6]}, ALBERT with k-fold cross-validation [7]} and multilingual-BERT model with convolution neural networks (CNN) [8]}. All models scored the same averaged-weighted F1-score of 0.93.
Among the vast transformer-based models, Balouchzahi et al. (2021) [9]} preferred working on feature engineering for traditional machine learning. They combined char sequences with traditional word n-grams and syntactic word n-grams [10]}, [11]} for English texts. Their proposed approach got an averaged-weighted F1-score of 0.92 in English, a performance comparable with that of transformers.
