@inproceedings{palomar-giner-etal-2024-curated,
    title = "A {CURATE}d {CAT}alog: Rethinking the Extraction of Pretraining Corpora for Mid-Resourced Languages",
    author = "Palomar-Giner, Jorge  and
      Saiz, Jose Javier  and
      Espu{\~n}a, Ferran  and
      Mina, Mario  and
      Da Dalt, Severino  and
      Llop, Joan  and
      Ostendorff, Malte  and
      Ortiz Suarez, Pedro  and
      Rehm, Georg  and
      Gonzalez-Agirre, Aitor  and
      Villegas, Marta",
    editor = "Calzolari, Nicoletta  and
      Kan, Min-Yen  and
      Hoste, Veronique  and
      Lenci, Alessandro  and
      Sakti, Sakriani  and
      Xue, Nianwen",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.31",
    pages = "335--349",
    abstract = "We present and describe two language resources in this paper: CATalog 1.0, the largest text corpus in Catalan to date, and CURATE (Corpus Utility for RAting TExt), a modular, parallelizable pipeline used for processing and scoring documents based on text quality that we have optimised to run in High Performance Cluster (HPC) environments. In the coming sections we describe our data preprocessing pipeline at length; traditional pipelines usually implement a set of binary filters such that a given document is either in or out. In our experience with Catalan, in lower-resource settings it is more practical to instead assign a document a soft score to allow for more flexible decision-making. We describe how the document score is calculated and highlight its interpretability by showing that it is significantly correlated with human judgements as obtained from a comparative judgement experiment. We additionally describe the different subcorpora that make up CATalog 1.0.",
}

We present and describe two language resources in this paper: CATalog 1.0, the largest text corpus in Catalan to date, and CURATE (Corpus Utility for RAting TExt), a modular, parallelizable pipeline used for processing and scoring documents based on text quality that we have optimised to run in High Performance Cluster (HPC) environments. In the coming sections we describe our data preprocessing pipeline at length; traditional pipelines usually implement a set of binary filters such that a given document is either in or out. In our experience with Catalan, in lower-resource settings it is more practical to instead assign a document a soft score to allow for more flexible decision-making. We describe how the document score is calculated and highlight its interpretability by showing that it is significantly correlated with human judgements as obtained from a comparative judgement experiment. We additionally describe the different subcorpora that make up CATalog 1.0.
Palomar-Giner, Jorge Saiz, Jose Javier Espu{\~n}a, Ferran Mina, Mario Da Dalt, Severino Llop, Joan Ostendorff, Malte Ortiz Suarez, Pedro Rehm, Georg Gonzalez-Agirre, Aitor Villegas, Marta
A CURATEd CATalog: Rethinking the Extraction of Pretraining Corpora for Mid-Resourced Languages