@inproceedings{ekgren-etal-2024-gpt,
    title = "{GPT}-{SW}3: An Autoregressive Language Model for the {S}candinavian Languages",
    author = {Ekgren, Ariel  and
      Cuba Gyllensten, Amaru  and
      Stollenwerk, Felix  and
      {\"O}hman, Joey  and
      Isbister, Tim  and
      Gogoulou, Evangelia  and
      Carlsson, Fredrik  and
      Casademont, Judit  and
      Sahlgren, Magnus},
    editor = "Calzolari, Nicoletta  and
      Kan, Min-Yen  and
      Hoste, Veronique  and
      Lenci, Alessandro  and
      Sakti, Sakriani  and
      Xue, Nianwen",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.695",
    pages = "7886--7900",
    abstract = "This paper details the process of developing the first native large generative language model for the North Germanic languages, GPT-SW3. We cover all parts of the development process, from data collection and processing, training configuration and instruction finetuning, to evaluation, applications, and considerations for release strategies. We discuss pros and cons of developing large language models for smaller languages and in relatively peripheral regions of the globe, and we hope that this paper can serve as a guide and reference for other researchers that undertake the development of large generative models for smaller languages.",
}

This paper details the process of developing the first native large generative language model for the North Germanic languages, GPT-SW3. We cover all parts of the development process, from data collection and processing, training configuration and instruction finetuning, to evaluation, applications, and considerations for release strategies. We discuss pros and cons of developing large language models for smaller languages and in relatively peripheral regions of the globe, and we hope that this paper can serve as a guide and reference for other researchers that undertake the development of large generative models for smaller languages.
Ekgren, Ariel Cuba Gyllensten, Amaru Stollenwerk, Felix {\"O}hman, Joey Isbister, Tim Gogoulou, Evangelia Carlsson, Fredrik Casademont, Judit Sahlgren, Magnus
GPT-SW3: An Autoregressive Language Model for the Scandinavian Languages