While final accuracy provides a good measure of performance on the pre-training objective, LLMs' abilities are typically judged by their performance on evaluation tasks. With the caveat that we use base models, that is our models have not been instruction-tuned, fine-tuned, or adapted to human preferences in any way, we present their evaluation on popular benchmarks in this section. Furthermore, we also provide a qualitative evaluation of German-trained models. We refer the reader to Sec. 5.4 of the main manuscript and Sec. A.5 of the appendix for a more detailed description of the chosen evaluation tasks.