task_ids:language-modeling annotations_creators:no-annotation language_creators:found multilinguality:multilingual size_categories:2G<n<1B source_datasets:original language:en language:nb language:no language:nn language:sv language:da language:is language:fo license:other arxiv:2104.09617 region:us
\\nNorwegian Colossal Corpus v2. Short sequences of maximum 100k characters.
@inproceedings{kummervold-etal-2021-operationalizing,
    title = "Operationalizing a National Digital Library: The Case for a {N}orwegian Transformer Model",
    author = "Kummervold, Per E  and
      De la Rosa, Javier  and
      Wetjen, Freddy  and
      Brygfjeld, Svein Arne",
    booktitle = "Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa)",
    month = may # " 31--2 " # jun,
    year = "2021",
    address = "Reykjavik, Iceland (Online)",
    publisher = {Link{\"o}ping University Electronic Press, Sweden},
    url = "https://aclanthology.org/2021.nodalida-main.3",
    pages = "20--29",
    abstract = "In this work, we show the process of building a large-scale training set from digital and digitized collections at a national library. The resulting Bidirectional Encoder Representations from Transformers (BERT)-based language model for Norwegian outperforms multilingual BERT (mBERT) models in several token and sequence classification tasks for both Norwegian Bokm{\aa}l and Norwegian Nynorsk. Our model also improves the mBERT performance for other languages present in the corpus such as English, Swedish, and Danish. For languages not included in the corpus, the weights degrade moderately while keeping strong multilingual properties. Therefore, we show that building high-quality models within a memory institution using somewhat noisy optical character recognition (OCR) content is feasible, and we hope to pave the way for other memory institutions to follow.",
}
---
YAML tags:

annotations_creators:
- no-annotation
language_creators:
- found
language:
- en
- nb
- no
- nn
- sv
- da
- is
- fo
license:
- other
multilinguality:
- multilingual
pretty_name: NCC
size_categories:
- 2G<n<1B
source_datasets:
- original
task_categories:
- sequence-modeling
task_ids:
- language-modeling
extra_gated_prompt: "The Directive on Copyright in the Digital Single Market, which came into force on June 6 2019, amends the European Union copyright and database legislation and allows for Text and Data Mining (TDM) activities for research organizations and cultural heritage institutions. Under the terms of the aforementioned directive, by clicking on 'Access repository' you agree on using the text and data contained in this dataset for non-commercial scientific purposes only."

---
# Dataset Card for NbAiLab/NCC


## Table of Contents
- [Dataset Description](#dataset-description)
- [Dataset Summary](#dataset-summary)
- [Data Fields](#data-fiels)
- [Dataset Creation](#dataset-creation)
- [Statistics](#statistics)
- [Document Types](#document-types)
- [Languages](#languages)
- [Publish Periode](#publish-periode)
- [Considerations for Using the Data](#considerations-for-using-the-data)
- [Social Impact of Dataset](#social-impact-of-dataset)
- [Discussion of Biases](#discussion-of-biases)
- [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
- [Dataset Curators](#dataset-curators)
- [Licensing Information](#licensing-information)
- [Citation Information](#citation-information)

## Dataset Description
- **Homepage:** https://github.com/NbAiLab/notram
- **Repository:** https://github.com/NbAiLab/notram
- **Paper:** https://arxiv.org/abs/2104.09617
- **Point of Contact:** [Freddy Wetjen](mailto:freddy.wetjen@nb.no)

The Norwegian Colossal Corpus is a collection of multiple smaller Norwegian corpuses suitable for training large language models. We have done extensive cleaning on the datasets, and have made them available in a common format. The total size of the NCC is currently 45GB. 

## How to Use
```python
from datasets import load_dataset
data = load_dataset("NbAiLab/NCC", streaming=True)
```
## Download Data
If you do not want to use the HuggingFace Dataset-library for training, or if you want to do additional pre-processing, it is also possible to download the files locally.
```bash
# Clone the training set
git clone https://huggingface.co/datasets/NbAiLab/NCC

# Create one large training file of all shards without unpacking
cat NCC/data/train*.gz > onefile.json.gz
```

<details>
<summary>List of all the files.</summary>


* [train-shard-0001-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0001-of-0046.json.gz)
* [train-shard-0002-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0002-of-0046.json.gz)
* [train-shard-0003-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0003-of-0046.json.gz)
* [train-shard-0004-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0004-of-0046.json.gz)
* [train-shard-0005-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0005-of-0046.json.gz)
* [train-shard-0006-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0006-of-0046.json.gz)
* [train-shard-0007-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0007-of-0046.json.gz)
* [train-shard-0008-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0008-of-0046.json.gz)
* [train-shard-0009-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0009-of-0046.json.gz)
* [train-shard-0010-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0010-of-0046.json.gz)
* [train-shard-0011-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0011-of-0046.json.gz)
* [train-shard-0012-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0012-of-0046.json.gz)
* [train-shard-0013-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0013-of-0046.json.gz)
* [train-shard-0014-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0014-of-0046.json.gz)
* [train-shard-0015-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0015-of-0046.json.gz)
* [train-shard-0016-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0016-of-0046.json.gz)
* [train-shard-0017-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0017-of-0046.json.gz)
* [train-shard-0018-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0018-of-0046.json.gz)
* [train-shard-0019-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0019-of-0046.json.gz)
* [train-shard-0020-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0020-of-0046.json.gz)
* [train-shard-0021-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0021-of-0046.json.gz)
* [train-shard-0022-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0022-of-0046.json.gz)
* [train-shard-0023-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0023-of-0046.json.gz)
* [train-shard-0024-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0024-of-0046.json.gz)
* [train-shard-0025-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0025-of-0046.json.gz)
* [train-shard-0026-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0026-of-0046.json.gz)
* [train-shard-0027-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0027-of-0046.json.gz)
* [train-shard-0028-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0028-of-0046.json.gz)
* [train-shard-0029-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0029-of-0046.json.gz)
* [train-shard-0030-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0030-of-0046.json.gz)
* [train-shard-0031-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0031-of-0046.json.gz)
* [train-shard-0032-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0032-of-0046.json.gz)
* [train-shard-0033-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0033-of-0046.json.gz)
* [train-shard-0034-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0034-of-0046.json.gz)
* [train-shard-0035-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0035-of-0046.json.gz)
* [train-shard-0036-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0036-of-0046.json.gz)
* [train-shard-0037-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0037-of-0046.json.gz)
* [train-shard-0038-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0038-of-0046.json.gz)
* [train-shard-0039-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0039-of-0046.json.gz)
* [train-shard-0040-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0040-of-0046.json.gz)
* [train-shard-0041-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0041-of-0046.json.gz)
* [train-shard-0042-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0042-of-0046.json.gz)
* [train-shard-0043-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0043-of-0046.json.gz)
* [train-shard-0044-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0044-of-0046.json.gz)
* [train-shard-0045-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0045-of-0046.json.gz)
* [train-shard-0046-of-0046](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/train-shard-0046-of-0046.json.gz)
* [validation-shard-0001-of-0001](https://huggingface.co/datasets/NbAiLab/NCC/resolve/main/data/validation-shard-0001-of-0001.json.gz)


</details>

### Dataset Summary
The NCC dataset contains json lines with language training data. Here is an example json line:
```json
{
"id": "1006205",
"doc_type": "cc100",
"publish_year": 2021,
"lang_fasttext": "nn",
"lang_fasttext_conf": "0.641",
"text": "Eg har ein PLAN! KOS deg og ha ei fin helg"
}
```
## Data Fields
|**id:** | String with id to source of line and a unique identifier|
|:-----------|:------------|
|**doc_type** | String describing type of media text extracted from (I.e. book,newspaper etc)|
|**publish_year** | Integer. The year text published. When year is undetermined it is set to 2021.|
|**lang_fasttext** | String. Language of text identified by FastText|
|**lang_fasttext_conf** | String. Confidence calculated by FastText|
|**text** | String. The complete utf-8 document. If longer than 1M characters it is split.|

### Dataset Creation
We are providing a **train** and a **validation** split. The standard size of the validation is a single 1GB file, while train is sharded in 1GB chunks.
All files are gzipped.

Build date: 21012022

#### Initial Data Collection and Curation
The procedure for the dataset creation is described in detail in our paper.

### Summary
| Words         | Documents   |   Words/Document |
|--------------:|------------:|-----------------:|
| 6,905,570,165 | 20,830,348  |              331 |

### Document Types
| Source                                | Words         | Documents   | Words/Document   |
|--------------------------------------:|--------------:|------------:|-----------------:|
| newspaper_ocr                         | 1,974,452,883 | 9,872,470   | 199              |
| parliament                            | 1,273,353,169 | 9,321       | 136,611          |
| books                                 | 842,936,050   | 23,708      | 35,554           |
| newspapers_online_nb                  | 487,189,627   | 3,446,348   | 141              |
| maalfrid_regjeringen                  | 360,349,242   | 919,902     | 391              |
| maalfrid_ssb                          | 279,732,847   | 851,982     | 328              |
| maalfrid_uio                          | 181,916,296   | 771,480     | 235              |
| government_nb                         | 134,127,104   | 3,476       | 38,586           |
| wikipedia_download_nbo                | 110,845,615   | 523,593     | 211              |
| maalfrid_fylkesmannen                 | 102,849,898   | 463,021     | 222              |
| publicreports                         | 78,347,879    | 3,298       | 23,756           |
| maalfrid_nve                          | 66,656,315    | 301,966     | 220              |
| maalfrid_patentstyret                 | 64,985,154    | 213,991     | 303              |
| maalfrid_ntnu                         | 57,803,460    | 199,307     | 290              |
| newspapers_online_nn                  | 42,205,558    | 167,347     | 252              |
| lovdata_cd_odelsting_2005             | 36,370,948    | 1,933       | 18,815           |
| maalfrid_vegvesen                     | 33,431,887    | 166,203     | 201              |
| maalfrid_fhi                          | 32,784,098    | 144,363     | 227              |
| maalfrid_norad                        | 32,720,034    | 93,097      | 351              |
| maalfrid_skatteetaten                 | 32,567,691    | 82,589      | 394              |
| maalfrid_uib                          | 28,425,322    | 115,729     | 245              |
| wikipedia_download_nno                | 27,061,858    | 143,265     | 188              |
| maalfrid_forskningsradet              | 24,076,984    | 73,368      | 328              |
| maalfrid_nasjonalparkstyre            | 21,309,995    | 93,871      | 227              |
| government_nn                         | 18,316,345    | 1,063       | 17,230           |
| maalfrid_nmbu                         | 18,082,476    | 69,719      | 259              |
| maalfrid_oslomet                      | 17,710,771    | 47,022      | 376              |
| maalfrid_domstol                      | 16,678,270    | 51,038      | 326              |
| maalfrid_banenor                      | 16,445,420    | 70,360      | 233              |
| maalfrid_nav                          | 16,272,635    | 74,101      | 219              |
| maalfrid_landbruksdirektoratet        | 13,119,567    | 47,983      | 273              |
| maalfrid_helsedirektoratet            | 13,008,787    | 49,344      | 263              |
| maalfrid_nokut                        | 10,101,424    | 38,552      | 262              |
| maalfrid_hi                           | 10,046,751    | 39,065      | 257              |
| maalfrid_norges-bank                  | 9,924,489     | 37,171      | 266              |
| maalfrid_udir                         | 9,868,345     | 38,736      | 254              |
| maalfrid_vkm                          | 9,824,529     | 32,230      | 304              |
| maalfrid_nbim                         | 9,629,725     | 18,131      | 531              |
| maalfrid_miljodirektoratet            | 9,496,631     | 34,711      | 273              |
| maalfrid_distriktssenteret            | 9,375,506     | 38,525      | 243              |
| maalfrid_ngu                          | 9,231,905     | 34,619      | 266              |
| maalfrid_ptil                         | 9,214,434     | 34,250      | 269              |
| maalfrid_nord                         | 8,992,352     | 44,800      | 200              |
| maalfrid_fiskeridir                   | 8,297,897     | 33,446      | 248              |
| maalfrid_hivolda                      | 7,820,709     | 26,473      | 295              |
| maalfrid_difi                         | 7,789,290     | 35,733      | 217              |
| maalfrid_mattilsynet                  | 7,492,831     | 27,002      | 277              |
| maalfrid_havarikommisjonen            | 7,440,410     | 24,989      | 297              |
| maalfrid_kulturradet                  | 7,196,423     | 22,437      | 320              |
| maalfrid_ks                           | 6,915,503     | 27,439      | 252              |
| maalfrid_kystverket                   | 6,713,070     | 30,975      | 216              |
| maalfrid_udi                          | 6,433,540     | 19,134      | 336              |
| maalfrid_uia                          | 5,964,644     | 23,861      | 249              |
| maalfrid_hjelpemiddeldatabasen        | 5,892,662     | 34,192      | 172              |
| maalfrid_khrono                       | 5,859,186     | 19,970      | 293              |
| maalfrid_helsetilsynet                | 5,803,000     | 18,365      | 315              |
| maalfrid_moreforsk                    | 5,622,025     | 21,579      | 260              |
| maalfrid_jernbanedirektoratet         | 5,461,268     | 21,666      | 252              |
| maalfrid_veiviseren                   | 5,316,521     | 18,026      | 294              |
| lovdata_cd_somb_rundskriv_2005        | 5,264,746     | 3,215       | 1,637            |
| maalfrid_dsb                          | 5,199,259     | 17,814      | 291              |
| lovdata_cd_sentrale_forskrifter_2005  | 5,037,694     | 11,467      | 439              |
| maalfrid_husbanken                    | 4,711,069     | 15,053      | 312              |
| maalfrid_legemiddelverket             | 4,689,988     | 20,192      | 232              |
| maalfrid_vetinst                      | 4,674,951     | 14,492      | 322              |
| maalfrid_imdi                         | 4,636,355     | 15,290      | 303              |
| maalfrid_forsvarsbygg                 | 4,567,318     | 18,886      | 241              |
| maalfrid_sdir                         | 4,540,110     | 15,202      | 298              |
| maalfrid_konkurransetilsynet          | 4,512,807     | 12,617      | 357              |
| maalfrid_dsa                          | 4,498,837     | 15,898      | 282              |
| maalfrid_arkivverket                  | 4,493,280     | 16,515      | 272              |
| maalfrid_hiof                         | 4,473,731     | 23,119      | 193              |
| maalfrid_ehelse                       | 4,379,984     | 22,553      | 194              |
| maalfrid_inn                          | 4,326,704     | 26,277      | 164              |
| maalfrid_klagenemndssekretariatet     | 4,181,685     | 11,916      | 350              |
| maalfrid_sprakradet                   | 4,097,815     | 15,187      | 269              |
| maalfrid_dibk                         | 3,967,428     | 15,509      | 255              |
| maalfrid_nhh                          | 3,962,033     | 15,678      | 252              |
| maalfrid_kartverket                   | 3,732,184     | 18,710      | 199              |
| maalfrid_riksrevisjonen               | 3,680,555     | 10,922      | 336              |
| maalfrid_toll                         | 3,510,061     | 13,777      | 254              |
| maalfrid_nibio                        | 3,456,026     | 17,104      | 202              |
| maalfrid_met                          | 3,446,762     | 18,282      | 188              |
| maalfrid_bufdir                       | 3,354,740     | 11,470      | 292              |
| maalfrid_artsdatabanken               | 3,193,511     | 9,009       | 354              |
| maalfrid_politiet                     | 3,167,395     | 10,501      | 301              |
| maalfrid_nkom                         | 3,127,687     | 10,002      | 312              |
| maalfrid_vestlandfylke                | 3,060,166     | 12,075      | 253              |
| maalfrid_uis                          | 2,924,821     | 9,838       | 297              |
| maalfrid_sykkelbynettverket           | 2,820,702     | 11,818      | 238              |
| maalfrid_nlr                          | 2,646,014     | 15,851      | 166              |
| maalfrid_seniorporten                 | 2,616,054     | 8,111       | 322              |
| maalfrid_npd                          | 2,597,831     | 10,742      | 241              |
| maalfrid_aldringoghelse               | 2,430,767     | 6,788       | 358              |
| maalfrid_custompublish                | 2,430,747     | 9,184       | 264              |
| maalfrid_bioteknologiradet            | 2,393,891     | 5,996       | 399              |
| maalfrid_arbeidstilsynet              | 2,379,597     | 6,882       | 345              |
| maalfrid_nyemetoder                   | 2,376,468     | 10,771      | 220              |
| maalfrid_riksantikvaren               | 2,257,491     | 8,756       | 257              |
| maalfrid_sjt                          | 2,238,168     | 11,189      | 200              |
| lovdata_cd_lokaleforskrifter_2005     | 2,176,221     | 22,274      | 97               |
| maalfrid_hvl                          | 2,149,292     | 9,395       | 228              |
| maalfrid_luftfartstilsynet            | 2,101,272     | 9,866       | 212              |
| maalfrid_dfo                          | 2,073,203     | 9,165       | 226              |
| maalfrid_ldo                          | 2,047,969     | 7,299       | 280              |
| maalfrid_kompetansenorge              | 1,952,035     | 10,245      | 190              |
| maalfrid_forbrukerradet               | 1,945,089     | 7,330       | 265              |
| maalfrid_himolde                      | 1,913,699     | 9,975       | 191              |
| maalfrid_usn                          | 1,793,297     | 7,403       | 242              |
| lovdata_cd_norgeslover_2005           | 1,760,884     | 1,386       | 1,270            |
| maalfrid_naku                         | 1,754,510     | 5,239       | 334              |
| maalfrid_medietilsynet                | 1,608,424     | 6,611       | 243              |
| maalfrid_matematikksenteret           | 1,567,505     | 7,298       | 214              |
| maalfrid_forskningsetikk              | 1,545,336     | 5,545       | 278              |
| maalfrid_diku                         | 1,542,929     | 6,241       | 247              |
| maalfrid_godeidrettsanlegg            | 1,506,577     | 6,115       | 246              |
| maalfrid_dirmin                       | 1,467,255     | 5,303       | 276              |
| maalfrid_diskrimineringsnemnda        | 1,463,291     | 4,168       | 351              |
| maalfrid_naturfag                     | 1,450,662     | 5,976       | 242              |
| maalfrid_arbeidsretten                | 1,440,074     | 4,754       | 302              |
| lovdata_cd_rtv_rundskriv_2005         | 1,366,872     | 9,596       | 142              |
| maalfrid_fellesstudentsystem          | 1,359,292     | 10,321      | 131              |
| maalfrid_nupi                         | 1,286,395     | 5,491       | 234              |
| maalfrid_kriminalitetsforebygging     | 1,201,477     | 4,667       | 257              |
| maalfrid_anskaffelser                 | 1,187,544     | 5,479       | 216              |
| maalfrid_folketrygdfondet             | 1,183,502     | 4,253       | 278              |
| maalfrid_miljopakken                  | 1,170,252     | 5,513       | 212              |
| maalfrid_nih                          | 1,116,791     | 5,271       | 211              |
| maalfrid_statsbygg                    | 1,103,635     | 4,439       | 248              |
| lovdata_cd_skatt_rundskriv_2005       | 1,102,142     | 398         | 2,769            |
| maalfrid_nb                           | 1,055,200     | 4,135       | 255              |
| maalfrid_npolar                       | 1,051,181     | 2,653       | 396              |
| maalfrid_unit                         | 1,049,621     | 6,329       | 165              |
| maalfrid_valgdirektoratet             | 1,009,941     | 9,131       | 110              |
| maalfrid_barneombudet                 | 980,751       | 2,807       | 349              |
| maalfrid_datatilsynet                 | 974,679       | 2,965       | 328              |
| maalfrid_lottstift                    | 959,590       | 3,578       | 268              |
| maalfrid_aho                          | 953,568       | 4,528       | 210              |
| maalfrid_sykehuspartner               | 939,625       | 4,579       | 205              |
| maalfrid_naturfagsenteret             | 897,049       | 3,859       | 232              |
| maalfrid_khio                         | 849,973       | 3,377       | 251              |
| maalfrid_spesialenheten               | 824,209       | 2,127       | 387              |
| maalfrid_xn--miljlftet-o8ab           | 803,011       | 3,384       | 237              |
| maalfrid_samordnaopptak               | 792,595       | 2,368       | 334              |
| maalfrid_helsenorge                   | 780,465       | 3,034       | 257              |
| maalfrid_skrivesenteret               | 777,204       | 4,161       | 186              |
| maalfrid_mareano                      | 760,645       | 3,724       | 204              |
| maalfrid_fiskeridirektoratet          | 755,997       | 2,444       | 309              |
| maalfrid_sykehusinnkjop               | 738,720       | 4,340       | 170              |
| maalfrid_matportalen                  | 630,990       | 2,370       | 266              |
| maalfrid_spk                          | 613,180       | 2,152       | 284              |
| maalfrid_justervesenet                | 595,014       | 1,904       | 312              |
| maalfrid_pasientsikkerhetsprogrammet  | 594,399       | 4,684       | 126              |
| maalfrid_nhn                          | 579,713       | 3,581       | 161              |
| maalfrid_sshf                         | 572,570       | 1,897       | 301              |
| maalfrid_bibliotekutvikling           | 560,126       | 3,216       | 174              |
| maalfrid_nysgjerrigper                | 559,207       | 3,019       | 185              |
| maalfrid_nodnett                      | 538,021       | 2,689       | 200              |
| maalfrid_une                          | 513,586       | 1,255       | 409              |
| maalfrid_giek                         | 512,569       | 1,796       | 285              |
| maalfrid_samas                        | 501,177       | 2,548       | 196              |
| maalfrid_kriminalomsorgen             | 496,062       | 1,951       | 254              |
| maalfrid_kjonnsforskning              | 483,376       | 1,426       | 338              |
| maalfrid_kunstkultursenteret          | 470,009       | 1,435       | 327              |
| lovdata_cd_rundskriv_lovavdeling_2005 | 469,295       | 405         | 1,158            |
| maalfrid_nynorsksenteret              | 460,165       | 2,085       | 220              |
| maalfrid_ceres                        | 448,920       | 1,950       | 230              |
| maalfrid_stami                        | 445,031       | 1,160       | 383              |
| maalfrid_nsm                          | 442,110       | 1,536       | 287              |
| maalfrid_gjenopptakelse               | 420,205       | 1,467       | 286              |
| maalfrid_nfi                          | 420,128       | 1,523       | 275              |
| maalfrid_nidsenter                    | 410,785       | 1,631       | 251              |
| maalfrid_nasjonalmuseet               | 390,036       | 1,087       | 358              |
| maalfrid_forbrukertilsynet            | 387,579       | 1,227       | 315              |
| maalfrid_natursekken                  | 378,442       | 3,563       | 106              |
| maalfrid_fordelingsutvalget           | 355,121       | 1,385       | 256              |
| maalfrid_digdir                       | 349,548       | 2,105       | 166              |
| maalfrid_forsvaret                    | 331,183       | 1,215       | 272              |
| maalfrid_beccle                       | 329,568       | 1,517       | 217              |
| maalfrid_romsenter                    | 329,304       | 1,133       | 290              |
| maalfrid_geonorge                     | 301,869       | 1,622       | 186              |
| maalfrid_universell                   | 263,740       | 2,155       | 122              |
| maalfrid_ovf                          | 262,542       | 930         | 282              |
| maalfrid_forbrukereuropa              | 259,420       | 1,018       | 254              |
| maalfrid_politihogskolen              | 258,615       | 1,229       | 210              |
| maalfrid_vinmonopolet                 | 245,685       | 671         | 366              |
| maalfrid_energimerking                | 237,243       | 1,033       | 229              |
| maalfrid_ombudsmann                   | 225,947       | 418         | 540              |
| maalfrid_vea-fs                       | 224,712       | 1,261       | 178              |
| maalfrid_traumebevisst                | 224,297       | 2,417       | 92               |
| maalfrid_npe                          | 205,102       | 1,000       | 205              |
| maalfrid_pkh                          | 201,503       | 791         | 254              |
| maalfrid_helfo                        | 193,880       | 988         | 196              |
| maalfrid_opplaringslovutvalget        | 193,590       | 549         | 352              |
| maalfrid_regionaleforskningsfond      | 187,261       | 989         | 189              |
| maalfrid_nafkam                       | 177,295       | 571         | 310              |
| maalfrid_jernbanemagasinet            | 174,152       | 412         | 422              |
| maalfrid_polarhistorie                | 171,386       | 382         | 448              |
| maalfrid_aasentunet                   | 161,626       | 529         | 305              |
| maalfrid_riksteatret                  | 159,991       | 798         | 200              |
| maalfrid_realfagsloyper               | 157,166       | 748         | 210              |
| maalfrid_koro                         | 153,304       | 574         | 267              |
| maalfrid_squarespace                  | 146,931       | 504         | 291              |
| maalfrid_politietssikkerhetstjeneste  | 143,781       | 469         | 306              |
| maalfrid_unknown                      | 139,263       | 700         | 198              |
| maalfrid_whocc                        | 121,616       | 656         | 185              |
| maalfrid_konfliktraadet               | 120,258       | 372         | 323              |
| maalfrid_okokrim                      | 115,842       | 372         | 311              |
| maalfrid_brreg                        | 112,787       | 571         | 197              |
| maalfrid_riksmekleren                 | 110,737       | 558         | 198              |
| maalfrid_sismo                        | 110,700       | 309         | 358              |
| maalfrid_radetfordyreetikk            | 99,241        | 441         | 225              |
| maalfrid_akkreditert                  | 99,040        | 503         | 196              |
| maalfrid_sivilforsvaret               | 97,679        | 514         | 190              |
| maalfrid_lanekassen                   | 95,286        | 301         | 316              |
| maalfrid_digidel                      | 95,140        | 607         | 156              |
| maalfrid_generaladvokaten             | 91,385        | 294         | 310              |
| maalfrid_uit                          | 90,273        | 602         | 149              |
| maalfrid_nyinorge                     | 88,466        | 199         | 444              |
| maalfrid_lokforerskolen               | 87,224        | 468         | 186              |
| maalfrid_varsom                       | 85,382        | 563         | 151              |
| maalfrid_ffi                          | 80,137        | 220         | 364              |
| maalfrid_kulturminnefondet            | 79,767        | 411         | 194              |
| maalfrid_unesco                       | 76,951        | 382         | 201              |
| maalfrid_yrkesfisker                  | 74,807        | 501         | 149              |
| maalfrid_dekom                        | 72,148        | 1,307       | 55               |
| maalfrid_omsorgsforskning             | 71,675        | 321         | 223              |
| maalfrid_lektor2                      | 67,385        | 549         | 122              |
| maalfrid_openaccess                   | 63,554        | 192         | 331              |
| maalfrid_ssn                          | 63,036        | 302         | 208              |
| maalfrid_lokalhistorie                | 59,854        | 241         | 248              |
| maalfrid_nlb                          | 57,872        | 200         | 289              |
| maalfrid_riksadvokaten                | 57,563        | 155         | 371              |
| maalfrid_laudim                       | 57,500        | 393         | 146              |
| maalfrid_denkulturelleskolesekken     | 46,018        | 243         | 189              |
| maalfrid_sivilrett                    | 44,062        | 142         | 310              |
| maalfrid_htu                          | 43,330        | 169         | 256              |
| maalfrid_yr                           | 40,646        | 562         | 72               |
| maalfrid_informasjonskompetanse       | 40,351        | 330         | 122              |
| maalfrid_dep                          | 38,882        | 126         | 308              |
| maalfrid_finansportalen               | 38,506        | 180         | 213              |
| maalfrid_feide                        | 36,715        | 267         | 137              |
| maalfrid_kulturped                    | 36,013        | 96          | 375              |
| maalfrid_fug                          | 34,158        | 120         | 284              |
| maalfrid_kulturoghelse                | 33,424        | 184         | 181              |
| maalfrid_helseklage                   | 32,756        | 124         | 264              |
| maalfrid_nbsk                         | 30,674        | 211         | 145              |
| maalfrid_matogindustri                | 29,922        | 194         | 154              |
| maalfrid_sinn                         | 27,541        | 150         | 183              |
| maalfrid_transport21                  | 25,317        | 90          | 281              |
| maalfrid_konkursradet                 | 23,505        | 76          | 309              |
| maalfrid_vergemal                     | 23,271        | 77          | 302              |
| maalfrid_norec                        | 22,496        | 78          | 288              |
| maalfrid_pts                          | 20,459        | 78          | 262              |
| maalfrid_nasjonaleturistveger         | 19,922        | 110         | 181              |
| maalfrid_iearth                       | 19,281        | 146         | 132              |
| maalfrid_hjelpelinjen                 | 19,209        | 85          | 225              |
| maalfrid_russamtalen                  | 17,999        | 65          | 276              |
| maalfrid_xn--kvinneligomskjring-1ub   | 17,701        | 77          | 229              |
| maalfrid_nynorskbok                   | 17,600        | 96          | 183              |
| maalfrid_regjeringsadvokaten          | 17,416        | 55          | 316              |
| maalfrid_memu                         | 17,311        | 98          | 176              |
| maalfrid_xn--tilbakefring-2jb         | 15,814        | 49          | 322              |
| maalfrid_xn--forskerfr-t8a            | 15,724        | 172         | 91               |
| maalfrid_ringerikefengsel             | 15,669        | 28          | 559              |
| maalfrid_skeivtarkiv                  | 15,537        | 69          | 225              |
| maalfrid_samfunnskunnskap             | 15,110        | 60          | 251              |
| maalfrid_fordelingsutvalet            | 15,017        | 34          | 441              |
| maalfrid_skattefunn                   | 14,599        | 51          | 286              |
| maalfrid_shiprep                      | 14,165        | 142         | 99               |
| maalfrid_haldenfengsel                | 13,625        | 37          | 368              |
| maalfrid_sevuppt                      | 13,332        | 52          | 256              |
| maalfrid_forbrukerklageutvalget       | 12,698        | 49          | 259              |
| maalfrid_mhfa                         | 11,999        | 144         | 83               |
| maalfrid_ah                           | 11,787        | 36          | 327              |
| maalfrid_nettvett                     | 11,002        | 43          | 255              |
| maalfrid_uh-it                        | 10,828        | 273         | 39               |
| maalfrid_fishgen                      | 10,199        | 28          | 364              |
| maalfrid_designavgang                 | 10,164        | 75          | 135              |
| maalfrid_global                       | 9,051         | 41          | 220              |
| maalfrid_havmiljo                     | 8,607         | 68          | 126              |
| maalfrid_valg                         | 8,516         | 47          | 181              |
| maalfrid_miljoklagenemnda             | 7,797         | 35          | 222              |
| maalfrid_altinn                       | 7,695         | 49          | 157              |
| maalfrid_spinn-inn                    | 7,674         | 47          | 163              |
| maalfrid_kantinekurset                | 7,217         | 53          | 136              |
| maalfrid_bastoyfengsel                | 7,142         | 56          | 127              |
| maalfrid_norskpetroleum               | 6,083         | 119         | 51               |
| maalfrid_voldsoffererstatning         | 5,827         | 26          | 224              |
| maalfrid_musikkbasertmiljobehandling  | 5,186         | 39          | 132              |
| maalfrid_prosjektveiviseren           | 5,019         | 14          | 358              |
| maalfrid_aldersvennlig                | 4,919         | 32          | 153              |
| maalfrid_barentswatch                 | 4,829         | 32          | 150              |
| maalfrid_fmfiavo@fylkesmannen         | 4,702         | 68          | 69               |
| maalfrid_kk-utvalget                  | 4,697         | 19          | 247              |
| maalfrid_agropub                      | 4,434         | 17          | 260              |
| maalfrid_utdanningiverden             | 4,266         | 13          | 328              |
| maalfrid_overgangsbolig               | 3,769         | 35          | 107              |
| maalfrid_forsvaretsmuseer             | 3,706         | 34          | 109              |
| maalfrid_okopark                      | 3,282         | 12          | 273              |
| maalfrid_pst                          | 2,866         | 14          | 204              |
| maalfrid_sikkerhverdag                | 2,697         | 18          | 149              |
| maalfrid_arkitektur                   | 2,436         | 15          | 162              |
| maalfrid_velgekte                     | 2,287         | 10          | 228              |
| maalfrid_addlab                       | 2,109         | 12          | 175              |
| maalfrid_romerikefengsel              | 2,088         | 19          | 109              |
| maalfrid_utdanning                    | 2,009         | 12          | 167              |
| maalfrid_grunderskolen                | 1,994         | 7           | 284              |
| maalfrid_umb                          | 1,934         | 8           | 241              |
| maalfrid_oslofengsel                  | 1,756         | 8           | 219              |
| maalfrid_hjorteviltregisteret         | 1,600         | 5           | 320              |
| maalfrid_alleteller                   | 1,511         | 7           | 215              |
| maalfrid_webhuset                     | 1,409         | 5           | 281              |
| maalfrid_lykillinn                    | 1,349         | 4           | 337              |
| maalfrid_kulturfag                    | 1,215         | 6           | 202              |
| maalfrid_unimus                       | 940           | 4           | 235              |
| maalfrid_anleggsregisteret            | 928           | 5           | 185              |
| maalfrid_mangfoldsprisen              | 597           | 3           | 199              |
| maalfrid_algae2future                 | 456           | 8           | 57               |
| maalfrid_mammapresenterer             | 447           | 2           | 223              |
| maalfrid_karriereveiledning           | 391           | 27          | 14               |
| maalfrid_nodsms                       | 351           | 4           | 87               |
| maalfrid_kildekompasset               | 302           | 1           | 302              |
| maalfrid_praksisfou                   | 297           | 1           | 297              |
| maalfrid_retttilaalese                | 246           | 3           | 82               |
| maalfrid_indreostfoldfengsel          | 215           | 3           | 71               |
| maalfrid_xn--kroppsvingsforskning-gcc | 205           | 2           | 102              |
| maalfrid_pahoyden                     | 154           | 1           | 154              |
| maalfrid_norren                       | 42            | 1           | 42               |

### Languages
| Language   | Words         | Documents   | Words/Document   |
|-----------:|--------------:|------------:|-----------------:|
| no         | 5,050,752,505 | 17,177,223  | 294              |
| da         | 940,216,574   | 574,211     | 1,637            |
| en         | 474,855,361   | 1,526,795   | 311              |
| nn         | 299,753,996   | 987,701     | 303              |
| fr         | 49,409,701    | 108,071     | 457              |
| de         | 27,159,878    | 85,230      | 318              |
| sv         | 18,773,092    | 118,753     | 158              |
| es         | 10,057,791    | 42,177      | 238              |
| fi         | 8,104,322     | 46,710      | 173              |
| et         | 3,309,661     | 24,183      | 136              |
| cs         | 2,652,151     | 21,793      | 121              |
| pt         | 2,550,218     | 16,407      | 155              |
| oc         | 2,123,730     | 4,927       | 431              |
| nl         | 1,984,501     | 11,813      | 167              |
| zh         | 1,470,751     | 8,146       | 180              |
| uk         | 1,459,484     | 5,096       | 286              |
| ca         | 1,370,260     | 4,476       | 306              |
| it         | 1,293,230     | 8,479       | 152              |
| la         | 1,281,920     | 797         | 1,608            |
| ru         | 1,231,482     | 6,796       | 181              |
| pl         | 852,304       | 9,396       | 90               |
| eu         | 831,276       | 3,195       | 260              |
| hu         | 659,973       | 8,499       | 77               |
| fa         | 494,551       | 2,047       | 241              |
| ja         | 351,634       | 4,994       | 70               |
| is         | 309,422       | 1,207       | 256              |
| id         | 226,296       | 2,033       | 111              |
| ar         | 205,632       | 1,173       | 175              |
| sl         | 140,913       | 1,858       | 75               |
| vi         | 139,122       | 982         | 141              |
| so         | 128,303       | 592         | 216              |
| hr         | 124,033       | 1,081       | 114              |
| el         | 117,624       | 618         | 190              |
| lv         | 106,626       | 123         | 866              |
| tr         | 92,680        | 1,630       | 56               |
| ro         | 80,804        | 635         | 127              |
| sr         | 71,953        | 970         | 74               |
| lt         | 70,148        | 869         | 80               |
| gl         | 65,152        | 692         | 94               |
| war        | 56,369        | 274         | 205              |
| ko         | 56,057        | 1,006       | 55               |
| th         | 54,067        | 367         | 147              |
| am         | 44,818        | 317         | 141              |
| sk         | 39,416        | 1,000       | 39               |
| ml         | 35,575        | 156         | 228              |
| ceb        | 35,337        | 331         | 106              |
| sq         | 34,461        | 238         | 144              |
| tl         | 30,839        | 177         | 174              |
| kk         | 27,827        | 72          | 386              |
| eo         | 24,187        | 859         | 28               |
| mn         | 21,540        | 22          | 979              |
| sw         | 18,670        | 72          | 259              |
| pnb        | 18,403        | 80          | 230              |
| sh         | 17,807        | 213         | 83               |
| gu         | 16,973        | 13          | 1,305            |
| bg         | 16,495        | 100         | 164              |
| ur         | 15,650        | 169         | 92               |
| mk         | 13,305        | 65          | 204              |
| ckb        | 9,119         | 43          | 212              |
| ku         | 9,071         | 57          | 159              |
| ast        | 7,919         | 73          | 108              |
| az         | 7,907         | 59          | 134              |
| ms         | 7,051         | 483         | 14               |
| uz         | 6,924         | 56          | 123              |
| ta         | 4,180         | 60          | 69               |
| fy         | 3,841         | 68          | 56               |
| ga         | 3,761         | 174         | 21               |
| hy         | 3,456         | 43          | 80               |
| pa         | 3,299         | 17          | 194              |
| hi         | 2,783         | 39          | 71               |
| be         | 2,556         | 62          | 41               |
| bo         | 2,551         | 1           | 2,551            |
| ht         | 2,534         | 11          | 230              |
| jv         | 2,341         | 91          | 25               |
| min        | 2,206         | 18          | 122              |
| cy         | 2,052         | 52          | 39               |
| bs         | 2,047         | 66          | 31               |
| als        | 1,918         | 66          | 29               |
| su         | 1,888         | 29          | 65               |
| nds        | 1,869         | 162         | 11               |
| ps         | 1,832         | 15          | 122              |
| bn         | 1,797         | 22          | 81               |
| qu         | 1,498         | 14          | 107              |
| ilo        | 1,126         | 25          | 45               |
| mt         | 968           | 16          | 60               |
| si         | 942           | 29          | 32               |
| te         | 888           | 18          | 49               |
| my         | 784           | 15          | 52               |
| af         | 741           | 32          | 23               |
| io         | 715           | 15          | 47               |
| tt         | 684           | 22          | 31               |
| km         | 674           | 11          | 61               |
| br         | 645           | 40          | 16               |
| gn         | 638           | 11          | 58               |
| jbo        | 611           | 27          | 22               |
| as         | 584           | 2           | 292              |
| ug         | 581           | 6           | 96               |
| kv         | 562           | 3           | 187              |
| kn         | 544           | 22          | 24               |
| pam        | 480           | 2           | 240              |
| kw         | 475           | 19          | 25               |
| vep        | 419           | 34          | 12               |
| he         | 412           | 18          | 22               |
| ka         | 351           | 20          | 17               |
| yo         | 281           | 9           | 31               |
| wa         | 268           | 38          | 7                |
| ky         | 228           | 10          | 22               |
| azb        | 216           | 1           | 216              |
| ba         | 203           | 5           | 40               |
| gom        | 174           | 12          | 14               |
| ia         | 140           | 15          | 9                |
| mr         | 138           | 10          | 13               |
| lmo        | 134           | 27          | 4                |
| tg         | 129           | 3           | 43               |
| lb         | 115           | 26          | 4                |
| pms        | 115           | 16          | 7                |
| vec        | 67            | 3           | 22               |
| rue        | 67            | 2           | 33               |
| sco        | 61            | 6           | 10               |
| ie         | 59            | 11          | 5                |
| hsb        | 57            | 3           | 19               |
| ne         | 56            | 6           | 9                |
| bar        | 46            | 7           | 6                |
| cbk        | 46            | 2           | 23               |
| or         | 44            | 2           | 22               |
| mg         | 38            | 8           | 4                |
| os         | 36            | 3           | 12               |
| tk         | 36            | 4           | 9                |
| arz        | 31            | 1           | 31               |
| li         | 29            | 6           | 4                |
| gd         | 29            | 2           | 14               |
| eml        | 24            | 5           | 4                |
| diq        | 20            | 2           | 10               |
| lrc        | 20            | 1           | 20               |
| dsb        | 19            | 1           | 19               |
| yue        | 19            | 1           | 19               |
| nap        | 16            | 1           | 16               |
| nah        | 14            | 2           | 7                |
| wuu        | 14            | 1           | 14               |
| sd         | 14            | 1           | 14               |
| frr        | 13            | 3           | 4                |
| rm         | 12            | 2           | 6                |
| cv         | 12            | 1           | 12               |
| scn        | 9             | 2           | 4                |
| bh         | 8             | 1           | 8                |
| bcl        | 8             | 1           | 8                |
| co         | 7             | 1           | 7                |
| ce         | 4             | 1           | 4                |
| new        | 4             | 1           | 4                |
| vo         | 3             | 2           | 1                |
| mzn        | 3             | 1           | 3                |
| gv         | 3             | 1           | 3                |
| lo         | 2             | 1           | 2                |

### Publish Periode
|   Decade | Words         | Documents   | Words/Document   |
|---------:|--------------:|------------:|-----------------:|
|     2020 | 4,090,213,596 | 10,934,550  | 523              |
|     2010 | 355,391,417   | 2,415,563   | 1,511            |
|     2000 | 447,853,330   | 1,705,354   | 2,773            |
|     1990 | 767,392,364   | 2,513,364   | 3,051            |
|     1980 | 160,980,586   | 538,665     | 3,011            |
|     1970 | 186,113,674   | 829,646     | 2,222            |
|     1960 | 149,421,535   | 834,219     | 1,807            |
|     1950 | 97,863,608    | 478,628     | 2,041            |
|     1940 | 122,648,278   | 570,154     | 2,307            |
|     1930 | 35,635,053    | 697         | 508,420          |
|     1920 | 50,381,418    | 1,049       | 484,836          |
|     1910 | 62,599,984    | 1,221       | 504,678          |
|     1900 | 60,019,080    | 1,130       | 527,329          |
|     1890 | 86,781,861    | 1,777       | 485,878          |
|     1880 | 58,546,570    | 1,064       | 553,442          |
|     1870 | 26,492,662    | 632         | 407,191          |
|     1860 | 39,176,930    | 698         | 543,151          |
|     1850 | 53,801,490    | 846         | 634,038          |
|     1840 | 30,434,939    | 522         | 581,593          |
|     1830 | 18,189,838    | 368         | 481,719          |
|     1820 | 4,721,154     | 144         | 338,350          |
|     1810 | 910,798       | 57          | 124,880          |

## Considerations for Using the Data
This corpus contains data under copyright and is not allowed to be used outide the National Library of Norway. The dataset should not be distributed.

### Discussion of Biases
Please refer to our paper.

### Dataset Curators
[Freddy Wetjen](mailto:Freddy.wetjen@nb.no) and [Per Egil Kummervold](mailto:Per.Kummervold@nb.no)


## License
Various licences applies to different parts of the corpus. Every document in the corpus has a tag telling what **"doc_type"** it belongs to. If you are unable to accept any of the licenses, you should filter out the **"doc_type"** with a conflicting license. 

| Doc_type  | License  | 
| :-------- | :------------- |  
| government_nb, government_nn, parliament, publicreports, lovdata_cd_\*, maalfrid_\* | [NLOD 2.0](https://data.norge.no/nlod/en/2.0/)|
| newspapers_ocr, newspapers_pdf, books| [CC0 1.0](https://creativecommons.org/publicdomain/zero/1.0/)|
| newspapers_online_nb, newspapers_online_nn | [CC BY-NC 2.0](https://creativecommons.org/licenses/by-nc/2.0/)|
| opensubtitles, wikipedia | [CC BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/)
|

### Citation Information
We are preparing an article with detailed information about this corpus. Until it is published, please cite out paper discussing the first version of this corpus:
```
@inproceedings{kummervold-etal-2021-operationalizing,
title = {Operationalizing a National Digital Library: The Case for a {N}orwegian Transformer Model},
author = {Kummervold, Per E  and
De la Rosa, Javier  and
Wetjen, Freddy  and
Brygfjeld, Svein Arne",
booktitle = {Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa)},
year = "2021",
address = "Reykjavik, Iceland (Online)",
publisher = {Link{"o}ping University Electronic Press, Sweden},
url = "https://aclanthology.org/2021.nodalida-main.3",
pages = "20--29",
abstract = "In this work, we show the process of building a large-scale training set from digital and digitized collections at a national library.
The resulting Bidirectional Encoder Representations from Transformers (BERT)-based language model for Norwegian outperforms multilingual BERT (mBERT) models
in several token and sequence classification tasks for both Norwegian Bokm{aa}l and Norwegian Nynorsk. Our model also improves the mBERT performance for other
languages present in the corpus such as English, Swedish, and Danish. For languages not included in the corpus, the weights degrade moderately while keeping strong multilingual properties. Therefore,
we show that building high-quality models within a memory institution using somewhat noisy optical character recognition (OCR) content is feasible, and we hope to pave the way for other memory institutions to follow.",
}
```
