Our chosen transformer is a DistilBERT model that has been pre-trained on the Stanford Sentiment Treebank v2 (SST2) training dataset [1]}. We pair it with a DistilBERT auto-tokenizer, also pre-trained on SST2. DistilBERT models offer impressive performance while being computationally efficient, which is important for our task given its computational complexity. SST2 is a sentiment analysis dataset widely used as a performance benchmark and freely available from dozens of online repositories [2]}. Its training, test, and validation sets have 6920, 1821, and 872 examples respectively.
