Performance Comparison of Simple Transformer and Res-CNN-BiLSTM for Cyberbullying Classification
The task of text classification using Bidirectional based LSTM architectures
is computationally expensive and time consuming to train. For this,
transformers were discovered which effectively give good performance as
compared to the traditional deep learning architectures. In this paper we
present a performance based comparison between simple transformer based network
and Res-CNN-BiLSTM based network for cyberbullying text classification problem.
The results obtained show that transformer we trained with 0.65 million
parameters has significantly being able to beat the performance of
Res-CNN-BiLSTM with 48.82 million parameters for faster training speeds and
more generalized metrics. The paper also compares the 1-dimensional character
level embedding network and 100-dimensional glove embedding network with
transformer.