task_categories:translation annotations_creators:expert-generated language_creators:found language_creators:machine-generated language_creators:expert-generated multilinguality:translation size_categories:100K<n<1M language:de-cs language:de-en language:de-fr language:en-cs language:en-de language:en-fi language:en-gu language:en-kk language:en-lt language:en-ru language:en-zh language:fi-en language:fr-de language:gu-en language:kk-en language:lt-en language:ru-en language:zh-en license:unknown region:us
This shared task will examine automatic evaluation metrics for machine translation. We will provide
you with all of the translations produced in the translation task along with the human reference translations.
You will return your automatic metric scores for translations at the system-level and/or at the sentence-level.
We will calculate the system-level and sentence-level correlations of your scores with WMT19 human judgements
once the manual evaluation has been completed.
@inproceedings{ma-etal-2019-results,
    title = {Results of the WMT19 Metrics Shared Task: Segment-Level and Strong MT Systems Pose Big Challenges},
    author = {Ma, Qingsong and Wei, Johnny and Bojar, Ondřej and Graham, Yvette},
    booktitle = {Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1)},
    month = {aug},
    year = {2019},
    address = {Florence, Italy},
    publisher = {Association for Computational Linguistics},
    url = {https://aclanthology.org/W19-5302},
    doi = {10.18653/v1/W19-5302},
    pages = {62--90}
}
---
annotations_creators:
- expert-generated
language_creators:
- found
- machine-generated
- expert-generated
language:
- de-cs
- de-en
- de-fr
- en-cs
- en-de
- en-fi
- en-gu
- en-kk
- en-lt
- en-ru
- en-zh
- fi-en
- fr-de
- gu-en
- kk-en
- lt-en
- ru-en
- zh-en
license:
- unknown
multilinguality:
- translation
paperswithcode_id: null
pretty_name: WMT19 Metrics Shared Task
size_categories:
- 100K<n<1M
source_datasets: []
task_categories:
- translation
task_ids: []
---

# Dataset Card for WMT19 Metrics Task

## Table of Contents
- [Table of Contents](#table-of-contents)
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Homepage:** [WMT19 Metrics Shared Task](https://www.statmt.org/wmt19/metrics-task.html)
- **Repository:** [MT Metrics Eval Github Repository](https://github.com/google-research/mt-metrics-eval)
- **Paper:** [Paper](https://aclanthology.org/W19-5302/)

### Dataset Summary

[More Information Needed]

### Supported Tasks and Leaderboards

[More Information Needed]

### Languages



The dataset comprises the following language pairs:
- de-cs
- de-en
- de-fr
- en-cs
- en-de
- en-fi
- en-gu
- en-kk
- en-lt
- en-ru
- en-zh
- fi-en
- fr-de
- gu-en
- kk-en
- lt-en
- ru-en
- zh-en

## Dataset Structure

### Data Instances

[More Information Needed]

### Data Fields

[More Information Needed]

### Data Splits

[More Information Needed]

## Dataset Creation

### Curation Rationale

[More Information Needed]

### Source Data

#### Initial Data Collection and Normalization

[More Information Needed]

#### Who are the source language producers?

[More Information Needed]

### Annotations

#### Annotation process

[More Information Needed]

#### Who are the annotators?

[More Information Needed]

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

[More Information Needed]

### Citation Information

[More Information Needed]

### Contributions

Thanks to [@github-username](https://github.com/mustaszewski) for adding this dataset.
