We use XLM-RoBERTa because it achieves the best accuracy on the Russian section of the XNLI corpus. We also used the RuBERT model because it attained the highest accuracy on the RuSentiment classification dataset among all the monolingual models for the Russian language. In each experiment, we use the Adam optimiser and \(learning\_rate = 5 \cdot 10^{-5}\)  as advised in [1]}. But our computational resources are too limited to use a \(batch\_size\)  value proposed in [1]}, we set \(batch\_size=16\) . Another important thing in any task of text classification is tackling the long texts. We have to take into account the fact that both RuBERT and XLM-RoBERTa cannot process texts that are longer than 512 tokens. In our study, we just take the first 512 tokens, also known as head-only truncation method from [1]}.
