One of the main challenges of LLMs is the resource gap, as they are mainly pretrained on English corpus and have limited access to data from other languages. English dominates the field of NLP as an extremely high-resource language with the most raw text data from various domains, leaving few of the over 7000 languages of the world represented in the field Joshi et al. (2020). This creates a disparity in language models' capability to handle different languages. Previous findings indicate that LLMs have difficulty comprehending and generating non-English texts, particularly in low-resource languagesNguyen et al. (2023); Zhu et al. (2023); Huang et al. (2023). To address the resource gap, several solutions have been proposed or implemented by researchers and practitioners. One possible solution is to increase the amount of data available from various languages and fields, and make it accessible for pretraining and evaluating LLMs Lin et al. (2022); Chen et al. (2022); Cahyawijaya et al. (2023). However, this approach incurs significant computational expenses and the resource gap persists. Alternatively, multilingual language models trained on texts from different languages concurrently, such as mBERT Devlin et al. (2019) and XLM-R Conneau et al. (2020), have been introduced to bridge the gap effectively.