LLMs represent a disruptive technology that has received significant attention from the public and is widely used across societies speaking different languages. Therefore, ensuring a democratization of the technology across people of different languages will represent an important value. Our study highlights that neglecting multilingualism while training a tokenizer representing a core component required for training LLMs can cause severe disadvantages, such as increased training costs and decreased downstream performance, raising major ethical concerns. Furthermore, the increased training costs translate into an increased carbon footprint, which has an environmental impact. Our findings support an improved development and usage of this fundamental technology.