Best results for these tasks were achieved with the following parameters: two learning epochs, maximal sequence length of 300 (<cite>BERT-base</cite>, EDUBERT) and 512 for the distilled models, all other parameter values were equal to the ones used for pre-training.