Many well-known LLMs are capable of comprehending input and generating responses across different languages, thanks to their pretraining on a diverse mix of corpus from multiple languages. However, due to the imbalanced distribution of language resources, collecting extensive training data for all languages is nearly impossible [10]. Taking the representative LLM BLOOM [13] as an example, it has been pretrained on 46 natural languages. Yet, this number accounts for only \(0.66\%\) of the roughly \(7,000\) languages currently in use. Moreover, within the corpus of these 46 languages, there exists extreme imbalance, with the high-resource English texts being 2.8 million times more than that of the low-resource Chitumbka language. This is not an isolated case.