HuggingFace platform enables researchers to explore various open source tools, codes, and technologies in various domains, including, Image Processing, NLP, and other data science works. Several transformers from HuggingFacehttps://huggingface.co/ platform were employed and fine-tuned using SimpleTransformershttps://simpletransformers.ai/ library that provides a user-friendly API to initialize, train (or fine-tune) and evaluate a task-specific transformer model from HuggingFace. Six transformer-based models, namely: BERT [1]}, ALBERT [2]}, Roberta [3]}, DistilBERT [4]}, XLNet [5]}, and ELECTRA [6]}, are used in this paper. Maximum sequence length and learning rate were set to 100 and 3e-5, respectively, and the rest of the parameters were set as default. All models were fine-tuned for 15 epochs in 5 folds.
