This work represents a fundamental step to a better understanding of the impact of the tokenizer on the models' downstream performance. We have shown that training tokenizers with a balanced share across languages achieve comparable low fertility and parity scores across all languages, which has important implications. Higher fertility results in up to 68% more computational costs during training and prevents the model from learning long-range dependencies in limited context windows.