@inproceedings{choi-etal-2024-optimizing,
    title = "Optimizing Language Augmentation for Multilingual Large Language Models: A Case Study on {K}orean",
    author = "Choi, ChangSu  and
      Jeong, Yongbin  and
      Park, Seoyoon  and
      Won, Inho  and
      Lim, HyeonSeok  and
      Kim, SangMin  and
      Kang, Yejee  and
      Yoon, Chanhyuk  and
      Park, Jaewan  and
      Lee, Yiseul  and
      Lee, HyeJin  and
      Hahm, Younggyun  and
      Kim, Hansaem  and
      Lim, KyungTae",
    editor = "Calzolari, Nicoletta  and
      Kan, Min-Yen  and
      Hoste, Veronique  and
      Lenci, Alessandro  and
      Sakti, Sakriani  and
      Xue, Nianwen",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.1095",
    pages = "12514--12526",
    abstract = "Large language models (LLMs) use pretraining to predict the subsequent word; however, their expansion requires significant computing resources. Numerous big tech companies and research institutes have developed multilingual LLMs (MLLMs) to meet current demands, overlooking less-resourced languages (LRLs). This study proposed three strategies to enhance the performance of LRLs based on the publicly available MLLMs. First, the MLLM vocabularies of LRLs were expanded to enhance expressiveness. Second, bilingual data were used for pretraining to align the high- and less-resourced languages. Third, a high-quality small-scale instruction dataset was constructed and instruction-tuning was performed to augment the LRL. The experiments employed the Llama2 model and Korean was used as the LRL, which was quantitatively evaluated against other developed LLMs across eight tasks. Furthermore, a qualitative assessment was performed based on human evaluation and GPT4. Experimental results showed that our proposed Bllossom model exhibited superior performance in qualitative analyses compared to previously proposed Korean monolingual models.",
}

Large language models (LLMs) use pretraining to predict the subsequent word; however, their expansion requires significant computing resources. Numerous big tech companies and research institutes have developed multilingual LLMs (MLLMs) to meet current demands, overlooking less-resourced languages (LRLs). This study proposed three strategies to enhance the performance of LRLs based on the publicly available MLLMs. First, the MLLM vocabularies of LRLs were expanded to enhance expressiveness. Second, bilingual data were used for pretraining to align the high- and less-resourced languages. Third, a high-quality small-scale instruction dataset was constructed and instruction-tuning was performed to augment the LRL. The experiments employed the Llama2 model and Korean was used as the LRL, which was quantitatively evaluated against other developed LLMs across eight tasks. Furthermore, a qualitative assessment was performed based on human evaluation and GPT4. Experimental results showed that our proposed Bllossom model exhibited superior performance in qualitative analyses compared to previously proposed Korean monolingual models.
Choi, ChangSu Jeong, Yongbin Park, Seoyoon Won, Inho Lim, HyeonSeok Kim, SangMin Kang, Yejee Yoon, Chanhyuk Park, Jaewan Lee, Yiseul Lee, HyeJin Hahm, Younggyun Kim, Hansaem Lim, KyungTae
Optimizing Language Augmentation for Multilingual Large Language Models: A Case Study on Korean