Figure 1: Pretrained LLaMA models, which are primarily trained on English-dominated corpus (as depicted on the left), are not inherently proficient in handling non-English languages. We aim to investigate the necessity of vocabulary extension, further pretraining, and instruction tuning, as well as to what extent they influence the capability transfer. This exploration enables us to efficiently transfer LLaMA’s language capabilities to non-English languages (as illustrated on the right), minimizing costs in the process.