In the original paper on BERT [1]}, there are some tips for training and fine-tuning the BERT architecture for text classification given by the authors. The paper recommends using 2-4 epochs for training or fine-tuning and claims that using more epochs leads to over-fitting. It coincides completely with our experiment on the optimal number of epochs. But, since our training and testing data are in Russian, we use RuBERT and XLM-RoBERTa instead of the base English BERT. Thus, we manage to approve the pieces of advice from [1]} and show that they are valid for Russian and multilingual BERT-based models.
