language:ca license:cc0-1.0 region:us
---
language: ca
license: cc0-1.0
extra_gated_prompt: 'By accessing this dataset, you agree to abide by the BigScience
  Ethical Charter. The charter can be found at:

  https://hf.co/spaces/bigscience/ethical-charter'
extra_gated_fields:
  I have read and agree to abide by the BigScience Ethical Charter: checkbox
---
ROOTS Subset: roots_ca_catalan_government_crawling

# Catalan Government Crawling

- Dataset uid: `catalan_government_crawling`

### Description

The Catalan Government Crawling Corpus is a 39-million-token web corpus of Catalan built from the web. It has been obtained by crawling the .gencat domain and subdomains, belonging to the Catalan Government during September and October 2020. It consists of 39.117.909 tokens, 1.565.433 sentences and 71.043 documents. Documents are separated by single new lines. It is a subcorpus of the Catalan Textual Corpus.

### Homepage

https://zenodo.org/record/4636486

### Licensing

- open license
- cc0-1.0: Creative Commons Zero v1.0 Universal


### Speaker Locations

- Southern Europe
- Spain


### Sizes

- 0.0219 % of total
- 1.8426 % of ca

### BigScience processing steps

#### Filters applied to: ca

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_1024

