For individuals working with non-English languages, creating a LLM from scratch can be computationally intensive. As a result, many turn to adopt open-source, English-based pre-trained LLMs, such as BLOOM (Workshop et al., 2023), LLaMA2 (Touvron et al., 2023), and Mistral-7B (Jiang et al., 2023), as foundational models. Inspired by Ouyang et al.