task_categories:fill-mask annotations_creators:other language_creators:found multilinguality:multilingual size_categories:10M<n<100M source_datasets:original language:bg language:cs language:da language:de language:el language:en language:es language:et language:fi language:fr language:ga language:hu language:it language:lt language:lv language:mt language:nl language:pl language:pt language:ro language:sk language:sl language:sv license:cc-by-4.0 region:us
Legal-MC4: A Corpus Covering the Legal Part of MC4 for European Languages
---
annotations_creators:
- other
language_creators:
- found
language:
- bg 
- cs 
- da 
- de 
- el 
- en
- es 
- et 
- fi 
- fr 
- ga
- hu 
- it 
- lt 
- lv 
- mt
- nl 
- pl 
- pt 
- ro 
- sk 
- sl 
- sv
license:
- cc-by-4.0
multilinguality:
- multilingual
paperswithcode_id: null
pretty_name: "MC4_Legal: A Corpus Covering the Legal Part of MC4 for European Languages"
size_categories:
- 10M<n<100M
source_datasets:
- original
task_categories:
- fill-mask

---

# Dataset Card for MC4_Legal: A Corpus Covering the Legal Part of MC4 for European Languages

## Table of Contents

- [Table of Contents](#table-of-contents)
- [Dataset Description](#dataset-description)
    - [Dataset Summary](#dataset-summary)
    - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
    - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
    - [Data Instances](#data-instances)
    - [Data Fields](#data-fields)
    - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
    - [Curation Rationale](#curation-rationale)
    - [Source Data](#source-data)
    - [Annotations](#annotations)
    - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
    - [Social Impact of Dataset](#social-impact-of-dataset)
    - [Discussion of Biases](#discussion-of-biases)
    - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
    - [Dataset Curators](#dataset-curators)
    - [Licensing Information](#licensing-information)
    - [Citation Information](#citation-information)
    - [Contributions](#contributions)

## Dataset Description

- **Homepage:**
- **Repository:** [GitHub](https://github.com/JoelNiklaus/LegalDatasets/tree/main/pretrain/mc4_legal)
- **Paper:** 
- **Leaderboard:**
- **Point of Contact:** [Joel Niklaus](mailto:joel@niklaus.ai)

### Dataset Summary

This dataset contains large text resources (~106GB in total) from mc4 filtered for legal data that can be used for pretraining language models.

This dataset uses a different filtering method compared to [mc4_legal](https://huggingface.co/datasets/joelito/mc4_legal) and uses the smaller filtered [c4](https://huggingface.co/datasets/c4) dataset for the English split to speed up the filtering.

Use the dataset like this:
```python
from datasets import load_dataset
dataset = load_dataset("joelito/mc4_legal", "de", split='train', streaming=True)
```

### Supported Tasks and Leaderboards

The dataset supports the task of masked language modeling.

### Languages

The following languages are supported: bg, cs, da, de, el, en, es, et, fi, fr, ga, hu, it, lt, lv, mt, nl, pl, pt, ro, sk, sl, sv

## Dataset Structure

### Data Instances

The file format is jsonl.xz and there is a validation and train split available.

| Source   |   Size (MB) |       Words |   Documents |   Words/Document |
|:---------|------------:|------------:|------------:|-----------------:|
| all      |      448980 | 28599300521 |     9873288 |             2896 |
| bg       |          57 |     2390349 |         379 |             6306 |
| cs       |       31005 |  1840827375 |      677796 |             2715 |
| da       |         162 |    10466716 |        3231 |             3239 |
| de       |      105739 |  6184578784 |     3164461 |             1954 |
| el       |          30 |     1155977 |         307 |             3765 |
| en       |       13734 |   966539309 |      359283 |             2690 |
| es       |      132053 |  9058939804 |     2281888 |             3969 |
| et       |        2059 |   110198368 |       49987 |             2204 |
| fi       |        1270 |    62799074 |       44875 |             1399 |
| fr       |       30878 |  2117306229 |      598983 |             3534 |
| ga       |           1 |       32772 |           8 |             4096 |
| hu       |        4677 |   244911748 |       58857 |             4161 |
| it       |       46957 |  3053920779 |      990823 |             3082 |
| lt       |         156 |     9142223 |        1529 |             5979 |
| lv       |           1 |       58702 |          16 |             3668 |
| mt       |          65 |     3479869 |         731 |             4760 |
| nl       |         326 |    21962633 |        6875 |             3194 |
| pl       |       37950 |  2235839721 |      827641 |             2701 |
| pt       |       20120 |  1338147828 |      382173 |             3501 |
| ro       |        8816 |   551372510 |      136513 |             4038 |
| sk       |        5850 |   349265172 |      130701 |             2672 |
| sl       |        1742 |   107493024 |       32574 |             3299 |
| sv       |        5332 |   328471555 |      123657 |             2656 |


### Data Fields

[More Information Needed]

### Data Splits

#### Data Size

```bash
$ xz --list data/*.xz
Strms  Blocks   Compressed Uncompressed  Ratio  Check   Filename
    1       1  2,080.7 KiB     33.4 MiB  0.061  CRC64   data/bg.train.0.jsonl.xz
    1       1     22.8 KiB    315.9 KiB  0.072  CRC64   data/bg.validation.0.jsonl.xz
    1       1    608.0 MiB  3,881.0 MiB  0.157  CRC64   data/cs.train.0.jsonl.xz
    1       1    608.0 MiB  3,902.6 MiB  0.156  CRC64   data/cs.train.1.jsonl.xz
    1       1    256.1 MiB  1,644.5 MiB  0.156  CRC64   data/cs.train.2.jsonl.xz
    1       1  1,450.6 KiB  8,690.7 KiB  0.167  CRC64   data/cs.validation.0.jsonl.xz
    1       1  7,578.6 KiB     38.3 MiB  0.193  CRC64   data/da.train.0.jsonl.xz
    1       1     19.7 KiB     82.3 KiB  0.240  CRC64   data/da.validation.0.jsonl.xz
    1       1    608.0 MiB  3,026.9 MiB  0.201  CRC64   data/de.train.0.jsonl.xz
    1       1    608.0 MiB  3,038.7 MiB  0.200  CRC64   data/de.train.1.jsonl.xz
    1       1    608.0 MiB  3,036.1 MiB  0.200  CRC64   data/de.train.2.jsonl.xz
    1       1    608.0 MiB  3,040.3 MiB  0.200  CRC64   data/de.train.3.jsonl.xz
    1       1    608.0 MiB  3,038.6 MiB  0.200  CRC64   data/de.train.4.jsonl.xz
    1       1    608.0 MiB  3,044.2 MiB  0.200  CRC64   data/de.train.5.jsonl.xz
    1       1    608.0 MiB  3,043.8 MiB  0.200  CRC64   data/de.train.6.jsonl.xz
    1       1    608.0 MiB  3,038.2 MiB  0.200  CRC64   data/de.train.7.jsonl.xz
    1       1     55.1 MiB    274.7 MiB  0.201  CRC64   data/de.train.8.jsonl.xz
    1       1  5,033.5 KiB     24.5 MiB  0.201  CRC64   data/de.validation.0.jsonl.xz
    1       1  1,280.9 KiB     17.0 MiB  0.073  CRC64   data/el.train.0.jsonl.xz
    1       1      5,552 B     15.7 KiB  0.346  CRC64   data/el.validation.0.jsonl.xz
    1       1    608.0 MiB  2,602.1 MiB  0.234  CRC64   data/en.train.0.jsonl.xz
    1       1     90.0 MiB    386.5 MiB  0.233  CRC64   data/en.train.1.jsonl.xz
    1       1    826.6 KiB  3,298.8 KiB  0.251  CRC64   data/en.validation.0.jsonl.xz
    1       1    608.0 MiB  3,106.5 MiB  0.196  CRC64   data/es.train.0.jsonl.xz
    1       1    608.0 MiB  3,118.1 MiB  0.195  CRC64   data/es.train.1.jsonl.xz
    1       1    608.0 MiB  3,113.6 MiB  0.195  CRC64   data/es.train.2.jsonl.xz
    1       1    608.0 MiB  3,122.5 MiB  0.195  CRC64   data/es.train.3.jsonl.xz
    1       1    608.0 MiB  3,121.5 MiB  0.195  CRC64   data/es.train.4.jsonl.xz
    1       1    608.0 MiB  3,122.9 MiB  0.195  CRC64   data/es.train.5.jsonl.xz
    1       1    608.0 MiB  3,128.4 MiB  0.194  CRC64   data/es.train.6.jsonl.xz
    1       1    608.0 MiB  3,129.5 MiB  0.194  CRC64   data/es.train.7.jsonl.xz
    1       1    608.0 MiB  3,132.2 MiB  0.194  CRC64   data/es.train.8.jsonl.xz
    1       1    528.5 MiB  2,722.5 MiB  0.194  CRC64   data/es.train.9.jsonl.xz
    1       1  6,159.9 KiB     30.7 MiB  0.196  CRC64   data/es.validation.0.jsonl.xz
    1       1     93.5 MiB    506.2 MiB  0.185  CRC64   data/et.train.0.jsonl.xz
    1       1    136.2 KiB    571.3 KiB  0.238  CRC64   data/et.validation.0.jsonl.xz
    1       1     60.6 MiB    312.6 MiB  0.194  CRC64   data/fi.train.0.jsonl.xz
    1       1     63.2 KiB    262.4 KiB  0.241  CRC64   data/fi.validation.0.jsonl.xz
    1       1    608.0 MiB  3,400.7 MiB  0.179  CRC64   data/fr.train.0.jsonl.xz
    1       1    608.0 MiB  3,405.5 MiB  0.179  CRC64   data/fr.train.1.jsonl.xz
    1       1    135.9 MiB    763.7 MiB  0.178  CRC64   data/fr.train.2.jsonl.xz
    1       1  1,414.3 KiB  7,626.1 KiB  0.185  CRC64   data/fr.validation.0.jsonl.xz
    1       1     31.2 KiB    146.4 KiB  0.213  CRC64   data/ga.train.0.jsonl.xz
    1       0         32 B          0 B    ---  CRC64   data/ga.validation.0.jsonl.xz
    1       1    211.5 MiB  1,407.3 MiB  0.150  CRC64   data/hu.train.0.jsonl.xz
    1       1    212.9 KiB  1,287.6 KiB  0.165  CRC64   data/hu.validation.0.jsonl.xz
    1       1    608.0 MiB  2,963.4 MiB  0.205  CRC64   data/it.train.0.jsonl.xz
    1       1    608.0 MiB  2,970.0 MiB  0.205  CRC64   data/it.train.1.jsonl.xz
    1       1    608.0 MiB  2,973.7 MiB  0.204  CRC64   data/it.train.2.jsonl.xz
    1       1    315.2 MiB  1,541.6 MiB  0.204  CRC64   data/it.train.3.jsonl.xz
    1       1  2,419.3 KiB     11.2 MiB  0.211  CRC64   data/it.validation.0.jsonl.xz
    1       1  9,966.7 KiB     38.2 MiB  0.255  CRC64   data/lt.train.0.jsonl.xz
    1       1     17.2 KiB     84.7 KiB  0.203  CRC64   data/lt.validation.0.jsonl.xz
    1       1     66.4 KiB    326.7 KiB  0.203  CRC64   data/lv.train.0.jsonl.xz
    1       0         32 B          0 B    ---  CRC64   data/lv.validation.0.jsonl.xz
    1       1  2,851.6 KiB     16.7 MiB  0.167  CRC64   data/mt.train.0.jsonl.xz
    1       1      2,092 B      5,079 B  0.412  CRC64   data/mt.validation.0.jsonl.xz
    1       1     14.6 MiB     71.6 MiB  0.203  CRC64   data/nl.train.0.jsonl.xz
    1       1     23.5 KiB     79.2 KiB  0.296  CRC64   data/nl.validation.0.jsonl.xz
    1       1    608.0 MiB  3,635.5 MiB  0.167  CRC64   data/pl.train.0.jsonl.xz
    1       1    608.0 MiB  3,646.0 MiB  0.167  CRC64   data/pl.train.1.jsonl.xz
    1       1    401.9 MiB  2,409.0 MiB  0.167  CRC64   data/pl.train.2.jsonl.xz
    1       1  1,870.5 KiB     10.5 MiB  0.173  CRC64   data/pl.validation.0.jsonl.xz
    1       1    608.0 MiB  3,173.1 MiB  0.192  CRC64   data/pt.train.0.jsonl.xz
    1       1    329.1 MiB  1,721.6 MiB  0.191  CRC64   data/pt.train.1.jsonl.xz
    1       1    989.0 KiB  4,841.2 KiB  0.204  CRC64   data/pt.validation.0.jsonl.xz
    1       1    365.2 MiB  2,237.9 MiB  0.163  CRC64   data/ro.train.0.jsonl.xz
    1       1    419.2 KiB  2,320.4 KiB  0.181  CRC64   data/ro.validation.0.jsonl.xz
    1       1    266.1 MiB  1,668.1 MiB  0.160  CRC64   data/sk.train.0.jsonl.xz
    1       1    304.1 KiB  1,618.2 KiB  0.188  CRC64   data/sk.validation.0.jsonl.xz
    1       1     81.6 MiB    416.1 MiB  0.196  CRC64   data/sl.train.0.jsonl.xz
    1       1    101.0 KiB    416.6 KiB  0.242  CRC64   data/sl.validation.0.jsonl.xz
    1       1    252.0 MiB  1,423.2 MiB  0.177  CRC64   data/sv.train.0.jsonl.xz
    1       1    210.8 KiB  1,091.2 KiB  0.193  CRC64   data/sv.validation.0.jsonl.xz
-------------------------------------------------------------------------------
   74      72     20.0 GiB    106.2 GiB  0.189  CRC64   74 files
```

## Dataset Creation

The dataset was created by filtering mc4 for legal data. 
We used terms indicating legal citations to get the texts. 
Note that this dataset can be quite noisy, and the quality is not known.

### Curation Rationale

[More Information Needed]

### Source Data

#### Initial Data Collection and Normalization

[More Information Needed]

#### Who are the source language producers?

[More Information Needed]


### Annotations

#### Annotation process

[More Information Needed]

#### Who are the annotators?

[More Information Needed]

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

[More Information Needed]

### Citation Information

[More Information Needed]

### Contributions

Thanks to [@JoelNiklaus](https://github.com/joelniklaus) for adding this dataset.
