To measure the impact of our trained tokenizers on the model downstream performance, we trained one model for each tokenizer. In particular, for each of our 24 trained tokenizers, we trained a 2.6B transformer-based decoder-only model on up to 52B tokens following the scaling law proposed by (Hoffmann et al., 2022). Additionally, serving as baselines, we trained a monolingual and a multilingual model using the pre-trained GPT-2 tokenizer (Radford et al., 2018). All models have been trained based on the causal language modeling training objective.