Furthermore, we highlight that the tokenizer choice can significantly impact the model's downstream performance. We could show that the BPE algorithm applies well to mono- and multilingual settings. For English, we show that a vocabulary size of 33k is sufficient, whereas multilingual models based on our five considered languages require a up to three times larger vocabulary size. Moreover, we could show that the SentencePiece library outperforms the Huggingface tokenizer library.