task_categories:translation annotations_creators:expert-generated language_creators:expert-generated multilinguality:translation size_categories:1K<n<10K source_datasets:original language:bg language:cs language:da language:de language:el language:en language:es language:et language:fi language:fr language:hr language:hu language:is language:it language:lt language:lv language:mt language:nl language:no language:pl language:pt language:ro language:sk language:sl language:sv language:tr license:cc-by-4.0 region:us
In October 2012, the European Union's (EU) Directorate General for Education and Culture ( DG EAC) released a translation memory (TM), i.e. a collection of sentences and their professionally produced translations, in twenty-six languages. This resource bears the name EAC Translation Memory, short EAC-TM.

EAC-TM covers up to 26 languages: 22 official languages of the EU (all except Irish) plus Icelandic, Croatian, Norwegian and Turkish. EAC-TM thus contains translations from English into the following 25 languages: Bulgarian, Czech, Danish, Dutch, Estonian, German, Greek, Finnish, French, Croatian, Hungarian, Icelandic, Italian, Latvian, Lithuanian, Maltese, Norwegian, Polish, Portuguese, Romanian, Slovak, Slovenian, Spanish, Swedish and Turkish.

All documents and sentences were originally written in English (source language is English) and then translated into the other languages. The texts were translated by staff of the National Agencies of the Lifelong Learning and Youth in Action programmes. They are typically professionals in the field of education/youth and EU programmes. They are thus not professional translators, but they are normally native speakers of the target language.
@Article{Steinberger2014,
        author={Steinberger, Ralf
                and Ebrahim, Mohamed
                and Poulis, Alexandros
                and Carrasco-Benitez, Manuel
                and Schl{\"u}ter, Patrick
                and Przybyszewski, Marek
                and Gilbro, Signe},
        title={An overview of the European Union's highly multilingual parallel corpora},
        journal={Language Resources and Evaluation},
        year={2014},
        month={Dec},
        day={01},
        volume={48},
        number={4},
        pages={679-707},
        issn={1574-0218},
        doi={10.1007/s10579-014-9277-0},
        url={https://doi.org/10.1007/s10579-014-9277-0}
}
---
annotations_creators:
- expert-generated
language_creators:
- expert-generated
language:
- bg
- cs
- da
- de
- el
- en
- es
- et
- fi
- fr
- hr
- hu
- is
- it
- lt
- lv
- mt
- nl
- 'no'
- pl
- pt
- ro
- sk
- sl
- sv
- tr
license:
- cc-by-4.0
multilinguality:
- translation
size_categories:
- 1K<n<10K
source_datasets:
- original
task_categories:
- translation
task_ids: []
pretty_name: Europa Education and Culture Translation Memory (EAC-TM)
dataset_info:
- config_name: en2bg
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - bg
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 664252
    num_examples: 4061
  download_size: 3521416
  dataset_size: 664252
- config_name: en2cs
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - cs
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 365983
    num_examples: 3351
  download_size: 3521416
  dataset_size: 365983
- config_name: en2da
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - da
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 422079
    num_examples: 3757
  download_size: 3521416
  dataset_size: 422079
- config_name: en2de
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - de
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 579566
    num_examples: 4473
  download_size: 3521416
  dataset_size: 579566
- config_name: en2el
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - el
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 491346
    num_examples: 2818
  download_size: 3521416
  dataset_size: 491346
- config_name: en2es
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - es
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 555218
    num_examples: 4303
  download_size: 3521416
  dataset_size: 555218
- config_name: en2et
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - et
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 247284
    num_examples: 2270
  download_size: 3521416
  dataset_size: 247284
- config_name: en2fi
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - fi
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 150560
    num_examples: 1458
  download_size: 3521416
  dataset_size: 150560
- config_name: en2fr
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - fr
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 575579
    num_examples: 4476
  download_size: 3521416
  dataset_size: 575579
- config_name: en2hu
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - hu
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 454802
    num_examples: 3455
  download_size: 3521416
  dataset_size: 454802
- config_name: en2is
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - is
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 268194
    num_examples: 2206
  download_size: 3521416
  dataset_size: 268194
- config_name: en2it
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - it
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 270634
    num_examples: 2170
  download_size: 3521416
  dataset_size: 270634
- config_name: en2lt
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - lt
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 358844
    num_examples: 3386
  download_size: 3521416
  dataset_size: 358844
- config_name: en2lv
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - lv
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 437487
    num_examples: 3880
  download_size: 3521416
  dataset_size: 437487
- config_name: en2mt
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - mt
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 178675
    num_examples: 1722
  download_size: 3521416
  dataset_size: 178675
- config_name: en2nb
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - nb
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 85833
    num_examples: 642
  download_size: 3521416
  dataset_size: 85833
- config_name: en2nl
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - nl
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 188531
    num_examples: 1805
  download_size: 3521416
  dataset_size: 188531
- config_name: en2pl
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - pl
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 515976
    num_examples: 4027
  download_size: 3521416
  dataset_size: 515976
- config_name: en2pt
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - pt
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 422125
    num_examples: 3501
  download_size: 3521416
  dataset_size: 422125
- config_name: en2ro
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - ro
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 345468
    num_examples: 3159
  download_size: 3521416
  dataset_size: 345468
- config_name: en2sk
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - sk
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 306049
    num_examples: 2972
  download_size: 3521416
  dataset_size: 306049
- config_name: en2sl
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - sl
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 577524
    num_examples: 4644
  download_size: 3521416
  dataset_size: 577524
- config_name: en2sv
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - sv
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 304954
    num_examples: 2909
  download_size: 3521416
  dataset_size: 304954
- config_name: en2tr
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - tr
  - name: sentence_type
    dtype:
      class_label:
        names:
          '0': form_data
          '1': sentence_data
  splits:
  - name: train
    num_bytes: 328267
    num_examples: 3198
  download_size: 3521416
  dataset_size: 328267
---

# Dataset Card for Europa Education and Culture Translation Memory (EAC-TM)

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Homepage:** [https://ec.europa.eu/jrc/en/language-technologies/eac-translation-memory](https://ec.europa.eu/jrc/en/language-technologies/eac-translation-memory)
- **Paper:** [https://link.springer.com/article/10.1007/s10579-014-9277-0](https://link.springer.com/article/10.1007/s10579-014-9277-0)
- **Point of Contact:** [ralf.steinberg@jrc.ec.europa.eu](mailto:ralf.steinberg@jrc.ec.europa.eu)

### Dataset Summary

This dataset is a corpus of manually produced translations from english to up to 25 languages, released in 2012 by the European Union's Directorate General for Education and Culture (EAC).

To load a language pair that is not part of the config, just specify the language code as language pair. For example, if you want to translate Czech to Greek:

`dataset = load_dataset("europa_eac_tm", language_pair=("cs", "el"))`

### Supported Tasks and Leaderboards

- `text2text-generation`: the dataset can be used to train a model for `machine-translation`. Machine translation models are usually evaluated using metrics such as [BLEU](https://huggingface.co/metrics/bleu), [ROUGE](https://huggingface.co/metrics/rouge) or [SacreBLEU](https://huggingface.co/metrics/sacrebleu). You can use the [mBART](https://huggingface.co/facebook/mbart-large-cc25) model for this task. This task has active leaderboards which can be found at [https://paperswithcode.com/task/machine-translation](https://paperswithcode.com/task/machine-translation), which usually rank models based on [BLEU score](https://huggingface.co/metrics/bleu).

### Languages

The sentences in this dataset were originally written in English (source language is English) and then translated into the other languages. The sentences are extracted from electroniv forms: application and report forms for decentralised actions of EAC's Life-long Learning Programme (LLP) and the Youth in Action Programme. The contents in the electronic forms are technically split into two types: (a) the labels and contents of drop-down menus (referred to as 'Forms' Data) and (b) checkboxes (referred to as 'Reference Data').

The dataset contains traduction of English sentences or parts of sentences to Bulgarian, Czech, Danish, Dutch, Estonian, German, Greek, Finnish, French, Croatian, Hungarian, Icelandic, Italian, Latvian, Lithuanian, Maltese, Norwegian, Polish, Portuguese, Romanian, Slovak, Slovenian, Spanish, Swedish and Turkish.

Language codes:
- `bg`
- `cs`
- `da`
- `de`
- `el`
- `en`
- `es`
- `et`
- `fi`
- `fr`
- `hr`
- `hu`
- `is`
- `it`
- `lt`
- `lv`
- `mt`
- `nl`
- `no`
- `pl`
- `pt`
- `ro`
- `sk`
- `sl`
- `sv`
- `tr`

## Dataset Structure

### Data Instances

```
{
  "translation": {
    "en":"Sentence to translate",
    "<target_language>": "Phrase à traduire",
  },
  "sentence_type": 0
}
```

### Data Fields

- `translation`: Mapping of sentences to translate (in English) and translated sentences.

- `sentence_type`: Integer value, 0 if the sentence is a 'form data' (extracted from the labels and contents of drop-down menus of the source electronic forms) or 1 if the sentence is a 'reference data' (extracted from the electronic forms checkboxes).

### Data Splits

The data is not splitted (only the `train` split is available).

## Dataset Creation

### Curation Rationale

The EAC-TM is relatively small compared to the JRC-Acquis and to DGT-TM, but it has the advantage that it focuses on a very different domain, namely that of education and culture. Also, it includes translation units for the languages Croatian (HR), Icelandic (IS), Norwegian (Bokmål, NB or Norwegian, NO) and Turkish (TR).

### Source Data

#### Initial Data Collection and Normalization

EAC-TM was built in the context of translating electronic forms: application and report forms for decentralised actions of EAC's Life-long Learning Programme (LLP) and the Youth in Action Programme. All documents and sentences were originally written in English (source language is English) and then translated into the other languages.

The contents in the electronic forms are technically split into two types: (a) the labels and contents of drop-down menus (referred to as 'Forms' Data) and (b) checkboxes (referred to as 'Reference Data'). Due to the different types of data, the two collections are kept separate. For example, labels can be 'Country', 'Please specify your home country' etc., while examples for reference data are 'Germany', 'Basic/general programmes', 'Education and Culture' etc.

The data consists of translations carried out between the end of the year 2008 and July 2012.

#### Who are the source language producers?

The texts were translated by staff of the National Agencies of the Lifelong Learning and Youth in Action programmes. They are typically professionals in the field of education/youth and EU programmes. They are thus not professional translators, but they are normally native speakers of the target language.

### Annotations

#### Annotation process

Sentences were manually translated by humans.
#### Who are the annotators?

The texts were translated by staff of the National Agencies of the Lifelong Learning and Youth in Action programmes. They are typically professionals in the field of education/youth and EU programmes. They are thus not professional translators, but they are normally native speakers of the target language.

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

© European Union, 1995-2020

The Commission's reuse policy is implemented by the [Commission Decision of 12 December 2011 on the reuse of Commission documents](https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32011D0833).

Unless otherwise indicated (e.g. in individual copyright notices), content owned by the EU on this website is licensed under the [Creative Commons Attribution 4.0 International (CC BY 4.0) licence](http://creativecommons.org/licenses/by/4.0/). This means that reuse is allowed, provided appropriate credit is given and changes are indicated.

You may be required to clear additional rights if a specific content depicts identifiable private individuals or includes third-party works. To use or reproduce content that is not owned by the EU, you may need to seek permission directly from the rightholders. Software or documents covered by industrial property rights, such as patents, trade marks, registered designs, logos and names, are excluded from the Commission's reuse policy and are not licensed to you.

### Citation Information

```
@Article{Steinberger2014,
        author={Steinberger, Ralf
                and Ebrahim, Mohamed
                and Poulis, Alexandros
                and Carrasco-Benitez, Manuel
                and Schl{\"u}ter, Patrick
                and Przybyszewski, Marek
                and Gilbro, Signe},
        title={An overview of the European Union's highly multilingual parallel corpora},
        journal={Language Resources and Evaluation},
        year={2014},
        month={Dec},
        day={01},
        volume={48},
        number={4},
        pages={679-707},
        issn={1574-0218},
        doi={10.1007/s10579-014-9277-0},
        url={https://doi.org/10.1007/s10579-014-9277-0}
}
```

### Contributions

Thanks to [@SBrandeis](https://github.com/SBrandeis) for adding this dataset.