task_categories:translation annotations_creators:crowdsourced language_creators:found multilinguality:translation size_categories:n<1K source_datasets:extended|WMT 2020 News Translation Task language:az language:ba language:en language:kaa language:kk language:ky language:ru language:sah language:tr language:uz license:mit arxiv:2109.04593 region:us
A Large-Scale Study of Machine Translation in Turkic Languages
@inproceedings{mirzakhalov2021large,
  title={A Large-Scale Study of Machine Translation in Turkic Languages},
  author={Mirzakhalov, Jamshidbek and Babu, Anoop and Ataman, Duygu and Kariev, Sherzod and Tyers, Francis and Abduraufov, Otabek and Hajili, Mammad and Ivanova, Sardana and Khaytbaev, Abror and Laverghetta Jr, Antonio and others},
  booktitle={Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing},
  pages={5876--5890},
  year={2021}
}
---
annotations_creators:
- crowdsourced
language_creators:
- found
language:
- az
- ba
- en
- kaa
- kk
- ky
- ru
- sah
- tr
- uz
license:
- mit
multilinguality:
- translation
pretty_name: turkic_xwmt
size_categories:
- n<1K
task_categories:
- translation
task_ids: []
source_datasets:
- extended|WMT 2020 News Translation Task
dataset_info:
- config_name: az-ba
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - az
        - ba
  splits:
  - name: test
    num_bytes: 266801
    num_examples: 600
  download_size: 12862396
  dataset_size: 266801
- config_name: az-en
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - az
        - en
  splits:
  - name: test
    num_bytes: 181156
    num_examples: 600
  download_size: 12862396
  dataset_size: 181156
- config_name: az-kaa
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - az
        - kaa
  splits:
  - name: test
    num_bytes: 134071
    num_examples: 300
  download_size: 12862396
  dataset_size: 134071
- config_name: az-kk
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - az
        - kk
  splits:
  - name: test
    num_bytes: 203798
    num_examples: 500
  download_size: 12862396
  dataset_size: 203798
- config_name: az-ky
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - az
        - ky
  splits:
  - name: test
    num_bytes: 210549
    num_examples: 500
  download_size: 12862396
  dataset_size: 210549
- config_name: az-ru
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - az
        - ru
  splits:
  - name: test
    num_bytes: 262739
    num_examples: 600
  download_size: 12862396
  dataset_size: 262739
- config_name: az-sah
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - az
        - sah
  splits:
  - name: test
    num_bytes: 144198
    num_examples: 300
  download_size: 12862396
  dataset_size: 144198
- config_name: az-tr
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - az
        - tr
  splits:
  - name: test
    num_bytes: 162447
    num_examples: 500
  download_size: 12862396
  dataset_size: 162447
- config_name: az-uz
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - az
        - uz
  splits:
  - name: test
    num_bytes: 194231
    num_examples: 600
  download_size: 12862396
  dataset_size: 194231
- config_name: ba-az
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ba
        - az
  splits:
  - name: test
    num_bytes: 266801
    num_examples: 600
  download_size: 12862396
  dataset_size: 266801
- config_name: ba-en
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ba
        - en
  splits:
  - name: test
    num_bytes: 431223
    num_examples: 1000
  download_size: 12862396
  dataset_size: 431223
- config_name: ba-kaa
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ba
        - kaa
  splits:
  - name: test
    num_bytes: 168895
    num_examples: 300
  download_size: 12862396
  dataset_size: 168895
- config_name: ba-kk
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ba
        - kk
  splits:
  - name: test
    num_bytes: 374756
    num_examples: 700
  download_size: 12862396
  dataset_size: 374756
- config_name: ba-ky
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ba
        - ky
  splits:
  - name: test
    num_bytes: 268986
    num_examples: 500
  download_size: 12862396
  dataset_size: 268986
- config_name: ba-ru
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ba
        - ru
  splits:
  - name: test
    num_bytes: 568101
    num_examples: 1000
  download_size: 12862396
  dataset_size: 568101
- config_name: ba-sah
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ba
        - sah
  splits:
  - name: test
    num_bytes: 179022
    num_examples: 300
  download_size: 12862396
  dataset_size: 179022
- config_name: ba-tr
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ba
        - tr
  splits:
  - name: test
    num_bytes: 309455
    num_examples: 700
  download_size: 12862396
  dataset_size: 309455
- config_name: ba-uz
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ba
        - uz
  splits:
  - name: test
    num_bytes: 410874
    num_examples: 900
  download_size: 12862396
  dataset_size: 410874
- config_name: en-az
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - az
  splits:
  - name: test
    num_bytes: 181156
    num_examples: 600
  download_size: 12862396
  dataset_size: 181156
- config_name: en-ba
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - ba
  splits:
  - name: test
    num_bytes: 431223
    num_examples: 1000
  download_size: 12862396
  dataset_size: 431223
- config_name: en-kaa
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - kaa
  splits:
  - name: test
    num_bytes: 126304
    num_examples: 300
  download_size: 12862396
  dataset_size: 126304
- config_name: en-kk
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - kk
  splits:
  - name: test
    num_bytes: 274728
    num_examples: 700
  download_size: 12862396
  dataset_size: 274728
- config_name: en-ky
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - ky
  splits:
  - name: test
    num_bytes: 198854
    num_examples: 500
  download_size: 12862396
  dataset_size: 198854
- config_name: en-ru
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - ru
  splits:
  - name: test
    num_bytes: 422718
    num_examples: 1000
  download_size: 12862396
  dataset_size: 422718
- config_name: en-sah
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - sah
  splits:
  - name: test
    num_bytes: 136431
    num_examples: 300
  download_size: 12862396
  dataset_size: 136431
- config_name: en-tr
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - tr
  splits:
  - name: test
    num_bytes: 210144
    num_examples: 700
  download_size: 12862396
  dataset_size: 210144
- config_name: en-uz
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - uz
  splits:
  - name: test
    num_bytes: 278971
    num_examples: 900
  download_size: 12862396
  dataset_size: 278971
- config_name: kaa-az
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kaa
        - az
  splits:
  - name: test
    num_bytes: 134071
    num_examples: 300
  download_size: 12862396
  dataset_size: 134071
- config_name: kaa-ba
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kaa
        - ba
  splits:
  - name: test
    num_bytes: 168895
    num_examples: 300
  download_size: 12862396
  dataset_size: 168895
- config_name: kaa-en
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kaa
        - en
  splits:
  - name: test
    num_bytes: 126304
    num_examples: 300
  download_size: 12862396
  dataset_size: 126304
- config_name: kaa-kk
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kaa
        - kk
  splits:
  - name: test
    num_bytes: 160022
    num_examples: 300
  download_size: 12862396
  dataset_size: 160022
- config_name: kaa-ky
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kaa
        - ky
  splits:
  - name: test
    num_bytes: 163763
    num_examples: 300
  download_size: 12862396
  dataset_size: 163763
- config_name: kaa-ru
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kaa
        - ru
  splits:
  - name: test
    num_bytes: 168349
    num_examples: 300
  download_size: 12862396
  dataset_size: 168349
- config_name: kaa-sah
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kaa
        - sah
  splits:
  - name: test
    num_bytes: 177151
    num_examples: 300
  download_size: 12862396
  dataset_size: 177151
- config_name: kaa-tr
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kaa
        - tr
  splits:
  - name: test
    num_bytes: 132055
    num_examples: 300
  download_size: 12862396
  dataset_size: 132055
- config_name: kaa-uz
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kaa
        - uz
  splits:
  - name: test
    num_bytes: 132789
    num_examples: 300
  download_size: 12862396
  dataset_size: 132789
- config_name: kk-az
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kk
        - az
  splits:
  - name: test
    num_bytes: 203798
    num_examples: 500
  download_size: 12862396
  dataset_size: 203798
- config_name: kk-ba
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kk
        - ba
  splits:
  - name: test
    num_bytes: 374756
    num_examples: 700
  download_size: 12862396
  dataset_size: 374756
- config_name: kk-en
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kk
        - en
  splits:
  - name: test
    num_bytes: 274728
    num_examples: 700
  download_size: 12862396
  dataset_size: 274728
- config_name: kk-kaa
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kk
        - kaa
  splits:
  - name: test
    num_bytes: 160022
    num_examples: 300
  download_size: 12862396
  dataset_size: 160022
- config_name: kk-ky
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kk
        - ky
  splits:
  - name: test
    num_bytes: 253421
    num_examples: 500
  download_size: 12862396
  dataset_size: 253421
- config_name: kk-ru
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kk
        - ru
  splits:
  - name: test
    num_bytes: 369633
    num_examples: 700
  download_size: 12862396
  dataset_size: 369633
- config_name: kk-sah
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kk
        - sah
  splits:
  - name: test
    num_bytes: 170149
    num_examples: 300
  download_size: 12862396
  dataset_size: 170149
- config_name: kk-tr
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kk
        - tr
  splits:
  - name: test
    num_bytes: 204442
    num_examples: 500
  download_size: 12862396
  dataset_size: 204442
- config_name: kk-uz
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - kk
        - uz
  splits:
  - name: test
    num_bytes: 290325
    num_examples: 700
  download_size: 12862396
  dataset_size: 290325
- config_name: ky-az
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ky
        - az
  splits:
  - name: test
    num_bytes: 210549
    num_examples: 500
  download_size: 12862396
  dataset_size: 210549
- config_name: ky-ba
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ky
        - ba
  splits:
  - name: test
    num_bytes: 268986
    num_examples: 500
  download_size: 12862396
  dataset_size: 268986
- config_name: ky-en
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ky
        - en
  splits:
  - name: test
    num_bytes: 198854
    num_examples: 500
  download_size: 12862396
  dataset_size: 198854
- config_name: ky-kaa
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ky
        - kaa
  splits:
  - name: test
    num_bytes: 163763
    num_examples: 300
  download_size: 12862396
  dataset_size: 163763
- config_name: ky-kk
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ky
        - kk
  splits:
  - name: test
    num_bytes: 253421
    num_examples: 500
  download_size: 12862396
  dataset_size: 253421
- config_name: ky-ru
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ky
        - ru
  splits:
  - name: test
    num_bytes: 265803
    num_examples: 500
  download_size: 12862396
  dataset_size: 265803
- config_name: ky-sah
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ky
        - sah
  splits:
  - name: test
    num_bytes: 173890
    num_examples: 300
  download_size: 12862396
  dataset_size: 173890
- config_name: ky-tr
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ky
        - tr
  splits:
  - name: test
    num_bytes: 168026
    num_examples: 400
  download_size: 12862396
  dataset_size: 168026
- config_name: ky-uz
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ky
        - uz
  splits:
  - name: test
    num_bytes: 209619
    num_examples: 500
  download_size: 12862396
  dataset_size: 209619
- config_name: ru-az
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ru
        - az
  splits:
  - name: test
    num_bytes: 262739
    num_examples: 600
  download_size: 12862396
  dataset_size: 262739
- config_name: ru-ba
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ru
        - ba
  splits:
  - name: test
    num_bytes: 568101
    num_examples: 1000
  download_size: 12862396
  dataset_size: 568101
- config_name: ru-en
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ru
        - en
  splits:
  - name: test
    num_bytes: 422718
    num_examples: 1000
  download_size: 12862396
  dataset_size: 422718
- config_name: ru-kaa
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ru
        - kaa
  splits:
  - name: test
    num_bytes: 168349
    num_examples: 300
  download_size: 12862396
  dataset_size: 168349
- config_name: ru-kk
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ru
        - kk
  splits:
  - name: test
    num_bytes: 369633
    num_examples: 700
  download_size: 12862396
  dataset_size: 369633
- config_name: ru-ky
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ru
        - ky
  splits:
  - name: test
    num_bytes: 265803
    num_examples: 500
  download_size: 12862396
  dataset_size: 265803
- config_name: ru-sah
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ru
        - sah
  splits:
  - name: test
    num_bytes: 178476
    num_examples: 300
  download_size: 12862396
  dataset_size: 178476
- config_name: ru-tr
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ru
        - tr
  splits:
  - name: test
    num_bytes: 304586
    num_examples: 700
  download_size: 12862396
  dataset_size: 304586
- config_name: ru-uz
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - ru
        - uz
  splits:
  - name: test
    num_bytes: 403551
    num_examples: 900
  download_size: 12862396
  dataset_size: 403551
- config_name: sah-az
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - sah
        - az
  splits:
  - name: test
    num_bytes: 144198
    num_examples: 300
  download_size: 12862396
  dataset_size: 144198
- config_name: sah-ba
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - sah
        - ba
  splits:
  - name: test
    num_bytes: 179022
    num_examples: 300
  download_size: 12862396
  dataset_size: 179022
- config_name: sah-en
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - sah
        - en
  splits:
  - name: test
    num_bytes: 136431
    num_examples: 300
  download_size: 12862396
  dataset_size: 136431
- config_name: sah-kaa
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - sah
        - kaa
  splits:
  - name: test
    num_bytes: 177151
    num_examples: 300
  download_size: 12862396
  dataset_size: 177151
- config_name: sah-kk
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - sah
        - kk
  splits:
  - name: test
    num_bytes: 170149
    num_examples: 300
  download_size: 12862396
  dataset_size: 170149
- config_name: sah-ky
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - sah
        - ky
  splits:
  - name: test
    num_bytes: 173890
    num_examples: 300
  download_size: 12862396
  dataset_size: 173890
- config_name: sah-ru
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - sah
        - ru
  splits:
  - name: test
    num_bytes: 178476
    num_examples: 300
  download_size: 12862396
  dataset_size: 178476
- config_name: sah-tr
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - sah
        - tr
  splits:
  - name: test
    num_bytes: 142182
    num_examples: 300
  download_size: 12862396
  dataset_size: 142182
- config_name: sah-uz
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - sah
        - uz
  splits:
  - name: test
    num_bytes: 142916
    num_examples: 300
  download_size: 12862396
  dataset_size: 142916
- config_name: tr-az
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - tr
        - az
  splits:
  - name: test
    num_bytes: 162447
    num_examples: 500
  download_size: 12862396
  dataset_size: 162447
- config_name: tr-ba
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - tr
        - ba
  splits:
  - name: test
    num_bytes: 309455
    num_examples: 700
  download_size: 12862396
  dataset_size: 309455
- config_name: tr-en
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - tr
        - en
  splits:
  - name: test
    num_bytes: 210144
    num_examples: 700
  download_size: 12862396
  dataset_size: 210144
- config_name: tr-kaa
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - tr
        - kaa
  splits:
  - name: test
    num_bytes: 132055
    num_examples: 300
  download_size: 12862396
  dataset_size: 132055
- config_name: tr-kk
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - tr
        - kk
  splits:
  - name: test
    num_bytes: 204442
    num_examples: 500
  download_size: 12862396
  dataset_size: 204442
- config_name: tr-ky
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - tr
        - ky
  splits:
  - name: test
    num_bytes: 168026
    num_examples: 400
  download_size: 12862396
  dataset_size: 168026
- config_name: tr-ru
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - tr
        - ru
  splits:
  - name: test
    num_bytes: 304586
    num_examples: 700
  download_size: 12862396
  dataset_size: 304586
- config_name: tr-sah
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - tr
        - sah
  splits:
  - name: test
    num_bytes: 142182
    num_examples: 300
  download_size: 12862396
  dataset_size: 142182
- config_name: tr-uz
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - tr
        - uz
  splits:
  - name: test
    num_bytes: 194761
    num_examples: 600
  download_size: 12862396
  dataset_size: 194761
- config_name: uz-az
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - uz
        - az
  splits:
  - name: test
    num_bytes: 194231
    num_examples: 600
  download_size: 12862396
  dataset_size: 194231
- config_name: uz-ba
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - uz
        - ba
  splits:
  - name: test
    num_bytes: 410874
    num_examples: 900
  download_size: 12862396
  dataset_size: 410874
- config_name: uz-en
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - uz
        - en
  splits:
  - name: test
    num_bytes: 278971
    num_examples: 900
  download_size: 12862396
  dataset_size: 278971
- config_name: uz-kaa
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - uz
        - kaa
  splits:
  - name: test
    num_bytes: 132789
    num_examples: 300
  download_size: 12862396
  dataset_size: 132789
- config_name: uz-kk
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - uz
        - kk
  splits:
  - name: test
    num_bytes: 290325
    num_examples: 700
  download_size: 12862396
  dataset_size: 290325
- config_name: uz-ky
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - uz
        - ky
  splits:
  - name: test
    num_bytes: 209619
    num_examples: 500
  download_size: 12862396
  dataset_size: 209619
- config_name: uz-ru
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - uz
        - ru
  splits:
  - name: test
    num_bytes: 403551
    num_examples: 900
  download_size: 12862396
  dataset_size: 403551
- config_name: uz-sah
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - uz
        - sah
  splits:
  - name: test
    num_bytes: 142916
    num_examples: 300
  download_size: 12862396
  dataset_size: 142916
- config_name: uz-tr
  features:
  - name: translation
    dtype:
      translation:
        languages:
        - uz
        - tr
  splits:
  - name: test
    num_bytes: 194761
    num_examples: 600
  download_size: 12862396
  dataset_size: 194761
config_names:
- az-ba
- az-en
- az-kaa
- az-kk
- az-ky
- az-ru
- az-sah
- az-tr
- az-uz
- ba-az
- ba-en
- ba-kaa
- ba-kk
- ba-ky
- ba-ru
- ba-sah
- ba-tr
- ba-uz
- en-az
- en-ba
- en-kaa
- en-kk
- en-ky
- en-ru
- en-sah
- en-tr
- en-uz
- kaa-az
- kaa-ba
- kaa-en
- kaa-kk
- kaa-ky
- kaa-ru
- kaa-sah
- kaa-tr
- kaa-uz
- kk-az
- kk-ba
- kk-en
- kk-kaa
- kk-ky
- kk-ru
- kk-sah
- kk-tr
- kk-uz
- ky-az
- ky-ba
- ky-en
- ky-kaa
- ky-kk
- ky-ru
- ky-sah
- ky-tr
- ky-uz
- ru-az
- ru-ba
- ru-en
- ru-kaa
- ru-kk
- ru-ky
- ru-sah
- ru-tr
- ru-uz
- sah-az
- sah-ba
- sah-en
- sah-kaa
- sah-kk
- sah-ky
- sah-ru
- sah-tr
- sah-uz
- tr-az
- tr-ba
- tr-en
- tr-kaa
- tr-kk
- tr-ky
- tr-ru
- tr-sah
- tr-uz
- uz-az
- uz-ba
- uz-en
- uz-kaa
- uz-kk
- uz-ky
- uz-ru
- uz-sah
- uz-tr
---

# Dataset Card for turkic_xwmt

## Table of Contents
- [Table of Contents](#table-of-contents)
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Homepage:** 
- **Repository:**[Github](https://github.com/turkic-interlingua/til-mt/tree/master/xwmt)
- **Paper:** [https://arxiv.org/abs/2109.04593](https://arxiv.org/abs/2109.04593)
- **Leaderboard:** [More Information Needed]
- **Point of Contact:** [turkicinterlingua@gmail.com](mailto:turkicinterlingua@gmail.com)

### Dataset Summary

To establish a comprehensive and challenging evaluation benchmark for Machine Translation in Turkic languages, we translate a test set originally introduced in WMT 2020 News Translation Task for English-Russian. The original dataset is profesionally translated and consists of sentences from news articles that are both English and Russian-centric. We adopt this evaluation set (X-WMT) and begin efforts to translate it into several Turkic languages. The current version of X-WMT includes covers 8 Turkic languages and 88 language directions with a minimum of 300 sentences per language direction.

### Supported Tasks and Leaderboards

[More Information Needed]

### Languages

Currently covered languages are (besides English and Russian):
- Azerbaijani (az)
- Bashkir (ba)
- Karakalpak (kaa)
- Kazakh (kk)
- Kirghiz (ky)
- Turkish (tr)
- Sakha (sah)
- Uzbek (uz)

## Dataset Structure

### Data Instances

A random example from the Russian-Uzbek set:
```
{"translation": {'ru': 'Моника Мутсвангва , министр информации Зимбабве , утверждает , что полиция вмешалась в отъезд Магомбейи из соображений безопасности и вследствие состояния его здоровья .', 'uz': 'Zimbabvening Axborot vaziri , Monika Mutsvanva Magombeyining xavfsizligi va sog'ligi tufayli bo'lgan jo'nab ketishinida politsiya aralashuvini ushlab turadi .'}}
```
### Data Fields

Each example has one field "translation" that contains two subfields: one per language, e.g. for the Russian-Uzbek set:
- **translation**: a dictionary with two subfields:
  - **ru**: the russian text
  - **uz**: the uzbek text
### Data Splits

<details>
  <summary>Click here to show the number of examples per configuration:</summary>
|         |   test |
|:--------|-------:|
| az-ba   |    600 |
| az-en   |    600 |
| az-kaa  |    300 |
| az-kk   |    500 |
| az-ky   |    500 |
| az-ru   |    600 |
| az-sah  |    300 |
| az-tr   |    500 |
| az-uz   |    600 |
| ba-az   |    600 |
| ba-en   |   1000 |
| ba-kaa  |    300 |
| ba-kk   |    700 |
| ba-ky   |    500 |
| ba-ru   |   1000 |
| ba-sah  |    300 |
| ba-tr   |    700 |
| ba-uz   |    900 |
| en-az   |    600 |
| en-ba   |   1000 |
| en-kaa  |    300 |
| en-kk   |    700 |
| en-ky   |    500 |
| en-ru   |   1000 |
| en-sah  |    300 |
| en-tr   |    700 |
| en-uz   |    900 |
| kaa-az  |    300 |
| kaa-ba  |    300 |
| kaa-en  |    300 |
| kaa-kk  |    300 |
| kaa-ky  |    300 |
| kaa-ru  |    300 |
| kaa-sah |    300 |
| kaa-tr  |    300 |
| kaa-uz  |    300 |
| kk-az   |    500 |
| kk-ba   |    700 |
| kk-en   |    700 |
| kk-kaa  |    300 |
| kk-ky   |    500 |
| kk-ru   |    700 |
| kk-sah  |    300 |
| kk-tr   |    500 |
| kk-uz   |    700 |
| ky-az   |    500 |
| ky-ba   |    500 |
| ky-en   |    500 |
| ky-kaa  |    300 |
| ky-kk   |    500 |
| ky-ru   |    500 |
| ky-sah  |    300 |
| ky-tr   |    400 |
| ky-uz   |    500 |
| ru-az   |    600 |
| ru-ba   |   1000 |
| ru-en   |   1000 |
| ru-kaa  |    300 |
| ru-kk   |    700 |
| ru-ky   |    500 |
| ru-sah  |    300 |
| ru-tr   |    700 |
| ru-uz   |    900 |
| sah-az  |    300 |
| sah-ba  |    300 |
| sah-en  |    300 |
| sah-kaa |    300 |
| sah-kk  |    300 |
| sah-ky  |    300 |
| sah-ru  |    300 |
| sah-tr  |    300 |
| sah-uz  |    300 |
| tr-az   |    500 |
| tr-ba   |    700 |
| tr-en   |    700 |
| tr-kaa  |    300 |
| tr-kk   |    500 |
| tr-ky   |    400 |
| tr-ru   |    700 |
| tr-sah  |    300 |
| tr-uz   |    600 |
| uz-az   |    600 |
| uz-ba   |    900 |
| uz-en   |    900 |
| uz-kaa  |    300 |
| uz-kk   |    700 |
| uz-ky   |    500 |
| uz-ru   |    900 |
| uz-sah  |    300 |
| uz-tr   |    600 |
</details>

## Dataset Creation

### Curation Rationale

[More Information Needed]

### Source Data

#### Initial Data Collection and Normalization

[More Information Needed]

#### Who are the source language producers?

[More Information Needed]

### Annotations

#### Annotation process

[More Information Needed]

#### Who are the annotators?

**Translators, annotators and dataset contributors** (in alphabetical order)

Abilxayr Zholdybai  
Aigiz Kunafin  
Akylbek Khamitov  
Alperen Cantez  
Aydos Muxammadiyarov  
Doniyorbek Rafikjonov  
Erkinbek Vokhabov  
Ipek Baris  
Iskander Shakirov  
Madina Zokirjonova  
Mohiyaxon Uzoqova  
Mukhammadbektosh Khaydarov  
Nurlan Maharramli  
Petr Popov  
Rasul Karimov  
Sariya Kagarmanova  
Ziyodabonu Qobiljon qizi 

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

[MIT License](https://github.com/turkic-interlingua/til-mt/blob/master/xwmt/LICENSE)

### Citation Information

```
@inproceedings{mirzakhalov2021large,
  title={A Large-Scale Study of Machine Translation in Turkic Languages},
  author={Mirzakhalov, Jamshidbek and Babu, Anoop and Ataman, Duygu and Kariev, Sherzod and Tyers, Francis and Abduraufov, Otabek and Hajili, Mammad and Ivanova, Sardana and Khaytbaev, Abror and Laverghetta Jr, Antonio and others},
  booktitle={Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing},
  pages={5876--5890},
  year={2021}
}
```

### Contributions

This project was carried out with the help and contributions from dozens of individuals and organizations. We acknowledge and greatly appreciate each and every one of them:

**Authors on the publications** (in alphabetical order)

Abror Khaytbaev  
Ahsan Wahab  
Aigiz Kunafin  
Anoop Babu  
Antonio Laverghetta Jr.  
Behzodbek Moydinboyev  
Dr. Duygu Ataman  
Esra Onal  
Dr. Francis Tyers  
Jamshidbek Mirzakhalov  
Dr. John Licato  
Dr. Julia Kreutzer  
Mammad Hajili  
Mokhiyakhon Uzokova  
Dr. Orhan Firat  
Otabek Abduraufov  
Sardana Ivanova  
Shaxnoza Pulatova  
Sherzod Kariev  
Dr. Sriram Chellappan  

**Translators, annotators and dataset contributors** (in alphabetical order)

Abilxayr Zholdybai  
Aigiz Kunafin  
Akylbek Khamitov  
Alperen Cantez  
Aydos Muxammadiyarov  
Doniyorbek Rafikjonov  
Erkinbek Vokhabov  
Ipek Baris  
Iskander Shakirov  
Madina Zokirjonova  
Mohiyaxon Uzoqova  
Mukhammadbektosh Khaydarov  
Nurlan Maharramli  
Petr Popov  
Rasul Karimov  
Sariya Kagarmanova  
Ziyodabonu Qobiljon qizi  

**Industry supporters**

[Google Cloud](https://cloud.google.com/solutions/education)  
[Khan Academy Oʻzbek](https://uz.khanacademy.org/)  
[The Foundation for the Preservation and Development of the Bashkir Language](https://bsfond.ru/)  

Thanks to [@mirzakhalov](https://github.com/mirzakhalov) for adding this dataset.