task_categories:question-answering task_ids:multiple-choice-qa annotations_creators:expert-generated language_creators:expert-generated multilinguality:monolingual size_categories:unknown source_datasets:extended|copa language:en license:cc-by-4.0 region:us
  XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning
The Cross-lingual Choice of Plausible Alternatives dataset is a benchmark to evaluate the ability of machine learning models to transfer commonsense reasoning across
languages. The dataset is the translation and reannotation of the English COPA (Roemmele et al. 2011) and covers 11 languages from 11 families and several areas around
the globe. The dataset is challenging as it requires both the command of world knowledge and the ability to generalise to new languages. All the details about the
creation of XCOPA and the implementation of the baselines are available in the paper.\n
  @article{ponti2020xcopa,
  title={{XCOPA: A} Multilingual Dataset for Causal Commonsense Reasoning},
  author={Edoardo M. Ponti, Goran Glava\v{s}, Olga Majewska, Qianchu Liu, Ivan Vuli\'{c} and Anna Korhonen},
  journal={arXiv preprint},
  year={2020},
  url={https://ducdauge.github.io/files/xcopa.pdf}
}

@inproceedings{roemmele2011choice,
  title={Choice of plausible alternatives: An evaluation of commonsense causal reasoning},
  author={Roemmele, Melissa and Bejan, Cosmin Adrian and Gordon, Andrew S},
  booktitle={2011 AAAI Spring Symposium Series},
  year={2011},
  url={https://people.ict.usc.edu/~gordon/publications/AAAI-SPRING11A.PDF},
}
---
annotations_creators:
- expert-generated
language_creators:
- expert-generated
language:
- en
license:
- cc-by-4.0
multilinguality:
- monolingual
pretty_name: XCOPA MT
size_categories:
- unknown
source_datasets:
- extended|copa
task_categories:
- question-answering
task_ids:
- multiple-choice-qa
paperswithcode_id: xcopa
dataset_info:
- config_name: nllb-200-distilled-600M
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 58092
    num_examples: 500
  - name: ht
    num_bytes: 58200
    num_examples: 500
  - name: it
    num_bytes: 59156
    num_examples: 500
  - name: id
    num_bytes: 59038
    num_examples: 500
  - name: qu
    num_bytes: 60464
    num_examples: 500
  - name: sw
    num_bytes: 58401
    num_examples: 500
  - name: zh
    num_bytes: 58016
    num_examples: 500
  - name: ta
    num_bytes: 60994
    num_examples: 500
  - name: th
    num_bytes: 56797
    num_examples: 500
  - name: tr
    num_bytes: 57256
    num_examples: 500
  - name: vi
    num_bytes: 56733
    num_examples: 500
  download_size: 1009631
  dataset_size: 643147
- config_name: nllb-200-distilled-1.3B
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 57531
    num_examples: 500
  - name: ht
    num_bytes: 57998
    num_examples: 500
  - name: it
    num_bytes: 58660
    num_examples: 500
  - name: id
    num_bytes: 58835
    num_examples: 500
  - name: qu
    num_bytes: 61138
    num_examples: 500
  - name: sw
    num_bytes: 58634
    num_examples: 500
  - name: zh
    num_bytes: 59319
    num_examples: 500
  - name: ta
    num_bytes: 60468
    num_examples: 500
  - name: th
    num_bytes: 56331
    num_examples: 500
  - name: tr
    num_bytes: 56979
    num_examples: 500
  - name: vi
    num_bytes: 56268
    num_examples: 500
  download_size: 1008646
  dataset_size: 642161
- config_name: nllb-200-1.3B
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 57282
    num_examples: 500
  - name: ht
    num_bytes: 57858
    num_examples: 500
  - name: it
    num_bytes: 58515
    num_examples: 500
  - name: id
    num_bytes: 58803
    num_examples: 500
  - name: qu
    num_bytes: 60172
    num_examples: 500
  - name: sw
    num_bytes: 58486
    num_examples: 500
  - name: zh
    num_bytes: 57671
    num_examples: 500
  - name: ta
    num_bytes: 60439
    num_examples: 500
  - name: th
    num_bytes: 55874
    num_examples: 500
  - name: tr
    num_bytes: 56806
    num_examples: 500
  - name: vi
    num_bytes: 56200
    num_examples: 500
  download_size: 1004579
  dataset_size: 638106
- config_name: nllb-200-3.3B
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 57660
    num_examples: 500
  - name: ht
    num_bytes: 58114
    num_examples: 500
  - name: it
    num_bytes: 58630
    num_examples: 500
  - name: id
    num_bytes: 58976
    num_examples: 500
  - name: qu
    num_bytes: 61276
    num_examples: 500
  - name: sw
    num_bytes: 58854
    num_examples: 500
  - name: zh
    num_bytes: 57851
    num_examples: 500
  - name: ta
    num_bytes: 60905
    num_examples: 500
  - name: th
    num_bytes: 56619
    num_examples: 500
  - name: tr
    num_bytes: 57071
    num_examples: 500
  - name: vi
    num_bytes: 56617
    num_examples: 500
  download_size: 1009049
  dataset_size: 642573
- config_name: xglm-564M
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 63358
    num_examples: 500
  - name: ht
    num_bytes: 64273
    num_examples: 500
  - name: it
    num_bytes: 70578
    num_examples: 500
  - name: id
    num_bytes: 63095
    num_examples: 500
  - name: qu
    num_bytes: 76634
    num_examples: 500
  - name: sw
    num_bytes: 68475
    num_examples: 500
  - name: zh
    num_bytes: 127703
    num_examples: 500
  - name: ta
    num_bytes: 109174
    num_examples: 500
  - name: th
    num_bytes: 71764
    num_examples: 500
  - name: tr
    num_bytes: 67498
    num_examples: 500
  - name: vi
    num_bytes: 69529
    num_examples: 500
  download_size: 1362468
  dataset_size: 852081
- config_name: xglm-1.7B
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 58674
    num_examples: 500
  - name: ht
    num_bytes: 57964
    num_examples: 500
  - name: it
    num_bytes: 59743
    num_examples: 500
  - name: id
    num_bytes: 58521
    num_examples: 500
  - name: qu
    num_bytes: 67219
    num_examples: 500
  - name: sw
    num_bytes: 60062
    num_examples: 500
  - name: zh
    num_bytes: 57233
    num_examples: 500
  - name: ta
    num_bytes: 64706
    num_examples: 500
  - name: th
    num_bytes: 59472
    num_examples: 500
  - name: tr
    num_bytes: 58155
    num_examples: 500
  - name: vi
    num_bytes: 57282
    num_examples: 500
  download_size: 1031393
  dataset_size: 659031
- config_name: xglm-2.9B
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 56815
    num_examples: 500
  - name: ht
    num_bytes: 59120
    num_examples: 500
  - name: it
    num_bytes: 60146
    num_examples: 500
  - name: id
    num_bytes: 60641
    num_examples: 500
  - name: qu
    num_bytes: 82619
    num_examples: 500
  - name: sw
    num_bytes: 60125
    num_examples: 500
  - name: zh
    num_bytes: 57593
    num_examples: 500
  - name: ta
    num_bytes: 67155
    num_examples: 500
  - name: th
    num_bytes: 60159
    num_examples: 500
  - name: tr
    num_bytes: 58299
    num_examples: 500
  - name: vi
    num_bytes: 57881
    num_examples: 500
  download_size: 1047842
  dataset_size: 680553
- config_name: xglm-4.5B
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 57355
    num_examples: 500
  - name: ht
    num_bytes: 62183
    num_examples: 500
  - name: it
    num_bytes: 59396
    num_examples: 500
  - name: id
    num_bytes: 57704
    num_examples: 500
  - name: qu
    num_bytes: 116554
    num_examples: 500
  - name: sw
    num_bytes: 59244
    num_examples: 500
  - name: zh
    num_bytes: 57123
    num_examples: 500
  - name: ta
    num_bytes: 70289
    num_examples: 500
  - name: th
    num_bytes: 58409
    num_examples: 500
  - name: tr
    num_bytes: 58127
    num_examples: 500
  - name: vi
    num_bytes: 57919
    num_examples: 500
  download_size: 1082379
  dataset_size: 714303
- config_name: xglm-7.5B
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 56766
    num_examples: 500
  - name: ht
    num_bytes: 57817
    num_examples: 500
  - name: it
    num_bytes: 58333
    num_examples: 500
  - name: id
    num_bytes: 57773
    num_examples: 500
  - name: qu
    num_bytes: 67010
    num_examples: 500
  - name: sw
    num_bytes: 58817
    num_examples: 500
  - name: zh
    num_bytes: 57227
    num_examples: 500
  - name: ta
    num_bytes: 62324
    num_examples: 500
  - name: th
    num_bytes: 55932
    num_examples: 500
  - name: tr
    num_bytes: 57305
    num_examples: 500
  - name: vi
    num_bytes: 56529
    num_examples: 500
  download_size: 1012936
  dataset_size: 645833
- config_name: bloom-560m
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 130778
    num_examples: 500
  - name: ht
    num_bytes: 118299
    num_examples: 500
  - name: it
    num_bytes: 95290
    num_examples: 500
  - name: id
    num_bytes: 60064
    num_examples: 500
  - name: qu
    num_bytes: 102968
    num_examples: 500
  - name: sw
    num_bytes: 146899
    num_examples: 500
  - name: zh
    num_bytes: 70813
    num_examples: 500
  - name: ta
    num_bytes: 86233
    num_examples: 500
  - name: th
    num_bytes: 155361
    num_examples: 500
  - name: tr
    num_bytes: 136837
    num_examples: 500
  - name: vi
    num_bytes: 61095
    num_examples: 500
  download_size: 1548970
  dataset_size: 1164637
- config_name: bloom-1b1
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 101964
    num_examples: 500
  - name: ht
    num_bytes: 91757
    num_examples: 500
  - name: it
    num_bytes: 74057
    num_examples: 500
  - name: id
    num_bytes: 56488
    num_examples: 500
  - name: qu
    num_bytes: 98982
    num_examples: 500
  - name: sw
    num_bytes: 87520
    num_examples: 500
  - name: zh
    num_bytes: 59371
    num_examples: 500
  - name: ta
    num_bytes: 74918
    num_examples: 500
  - name: th
    num_bytes: 128581
    num_examples: 500
  - name: tr
    num_bytes: 143310
    num_examples: 500
  - name: vi
    num_bytes: 55236
    num_examples: 500
  download_size: 1344990
  dataset_size: 972184
- config_name: bloom-1b7
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 85029
    num_examples: 500
  - name: ht
    num_bytes: 75448
    num_examples: 500
  - name: it
    num_bytes: 61350
    num_examples: 500
  - name: id
    num_bytes: 58084
    num_examples: 500
  - name: qu
    num_bytes: 77332
    num_examples: 500
  - name: sw
    num_bytes: 67131
    num_examples: 500
  - name: zh
    num_bytes: 57200
    num_examples: 500
  - name: ta
    num_bytes: 70436
    num_examples: 500
  - name: th
    num_bytes: 139759
    num_examples: 500
  - name: tr
    num_bytes: 100472
    num_examples: 500
  - name: vi
    num_bytes: 55737
    num_examples: 500
  download_size: 1219112
  dataset_size: 847978
- config_name: bloom-3b
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 73262
    num_examples: 500
  - name: ht
    num_bytes: 63961
    num_examples: 500
  - name: it
    num_bytes: 60275
    num_examples: 500
  - name: id
    num_bytes: 58006
    num_examples: 500
  - name: qu
    num_bytes: 89802
    num_examples: 500
  - name: sw
    num_bytes: 61519
    num_examples: 500
  - name: zh
    num_bytes: 56864
    num_examples: 500
  - name: ta
    num_bytes: 69482
    num_examples: 500
  - name: th
    num_bytes: 109418
    num_examples: 500
  - name: tr
    num_bytes: 120094
    num_examples: 500
  - name: vi
    num_bytes: 55980
    num_examples: 500
  download_size: 1187376
  dataset_size: 818663
- config_name: bloom-7b1
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 50296
    num_examples: 500
  - name: ht
    num_bytes: 53141
    num_examples: 500
  - name: it
    num_bytes: 59193
    num_examples: 500
  - name: id
    num_bytes: 56651
    num_examples: 500
  - name: qu
    num_bytes: 73218
    num_examples: 500
  - name: sw
    num_bytes: 58770
    num_examples: 500
  - name: zh
    num_bytes: 56282
    num_examples: 500
  - name: ta
    num_bytes: 61975
    num_examples: 500
  - name: th
    num_bytes: 82201
    num_examples: 500
  - name: tr
    num_bytes: 55094
    num_examples: 500
  - name: vi
    num_bytes: 55580
    num_examples: 500
  download_size: 1029650
  dataset_size: 662401
- config_name: llama-7B
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 57640
    num_examples: 500
  - name: ht
    num_bytes: 62634
    num_examples: 500
  - name: it
    num_bytes: 59497
    num_examples: 500
  - name: id
    num_bytes: 59138
    num_examples: 500
  - name: qu
    num_bytes: 71702
    num_examples: 500
  - name: sw
    num_bytes: 63238
    num_examples: 500
  - name: zh
    num_bytes: 59803
    num_examples: 500
  - name: ta
    num_bytes: 107865
    num_examples: 500
  - name: th
    num_bytes: 71665
    num_examples: 500
  - name: tr
    num_bytes: 58729
    num_examples: 500
  - name: vi
    num_bytes: 67266
    num_examples: 500
  download_size: 1106401
  dataset_size: 739177
- config_name: llama-13B
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 58524
    num_examples: 500
  - name: ht
    num_bytes: 58576
    num_examples: 500
  - name: it
    num_bytes: 59633
    num_examples: 500
  - name: id
    num_bytes: 57663
    num_examples: 500
  - name: qu
    num_bytes: 69152
    num_examples: 500
  - name: sw
    num_bytes: 63891
    num_examples: 500
  - name: zh
    num_bytes: 57540
    num_examples: 500
  - name: ta
    num_bytes: 85821
    num_examples: 500
  - name: th
    num_bytes: 55881
    num_examples: 500
  - name: tr
    num_bytes: 56783
    num_examples: 500
  - name: vi
    num_bytes: 55295
    num_examples: 500
  download_size: 1045868
  dataset_size: 678759
- config_name: llama-30B
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 55792
    num_examples: 500
  - name: ht
    num_bytes: 55836
    num_examples: 500
  - name: it
    num_bytes: 59578
    num_examples: 500
  - name: id
    num_bytes: 58384
    num_examples: 500
  - name: qu
    num_bytes: 60479
    num_examples: 500
  - name: sw
    num_bytes: 60740
    num_examples: 500
  - name: zh
    num_bytes: 57099
    num_examples: 500
  - name: ta
    num_bytes: 74192
    num_examples: 500
  - name: th
    num_bytes: 54577
    num_examples: 500
  - name: tr
    num_bytes: 55743
    num_examples: 500
  - name: vi
    num_bytes: 56371
    num_examples: 500
  download_size: 1015352
  dataset_size: 648791
- config_name: RedPajama-INCITE-Base-3B-v1
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 66862
    num_examples: 500
  - name: ht
    num_bytes: 67548
    num_examples: 500
  - name: it
    num_bytes: 60220
    num_examples: 500
  - name: id
    num_bytes: 58585
    num_examples: 500
  - name: qu
    num_bytes: 84898
    num_examples: 500
  - name: sw
    num_bytes: 78422
    num_examples: 500
  - name: zh
    num_bytes: 60708
    num_examples: 500
  - name: ta
    num_bytes: 99438
    num_examples: 500
  - name: th
    num_bytes: 83022
    num_examples: 500
  - name: tr
    num_bytes: 64835
    num_examples: 500
  - name: vi
    num_bytes: 68696
    num_examples: 500
  download_size: 1161592
  dataset_size: 793234
- config_name: RedPajama-INCITE-7B-Base
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 59722
    num_examples: 500
  - name: ht
    num_bytes: 54824
    num_examples: 500
  - name: it
    num_bytes: 59511
    num_examples: 500
  - name: id
    num_bytes: 59526
    num_examples: 500
  - name: qu
    num_bytes: 102986
    num_examples: 500
  - name: sw
    num_bytes: 69382
    num_examples: 500
  - name: zh
    num_bytes: 59507
    num_examples: 500
  - name: ta
    num_bytes: 88701
    num_examples: 500
  - name: th
    num_bytes: 65715
    num_examples: 500
  - name: tr
    num_bytes: 61684
    num_examples: 500
  - name: vi
    num_bytes: 65257
    num_examples: 500
  download_size: 1114614
  dataset_size: 746815
- config_name: open_llama_3b
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 66399
    num_examples: 500
  - name: ht
    num_bytes: 60389
    num_examples: 500
  - name: it
    num_bytes: 60711
    num_examples: 500
  - name: id
    num_bytes: 60704
    num_examples: 500
  - name: qu
    num_bytes: 91950
    num_examples: 500
  - name: sw
    num_bytes: 72466
    num_examples: 500
  - name: zh
    num_bytes: 62617
    num_examples: 500
  - name: ta
    num_bytes: 106600
    num_examples: 500
  - name: th
    num_bytes: 203185
    num_examples: 500
  - name: tr
    num_bytes: 66524
    num_examples: 500
  - name: vi
    num_bytes: 77933
    num_examples: 500
  download_size: 1439470
  dataset_size: 929478
- config_name: open_llama_7b
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 57157
    num_examples: 500
  - name: ht
    num_bytes: 54184
    num_examples: 500
  - name: it
    num_bytes: 59425
    num_examples: 500
  - name: id
    num_bytes: 57354
    num_examples: 500
  - name: qu
    num_bytes: 73290
    num_examples: 500
  - name: sw
    num_bytes: 65718
    num_examples: 500
  - name: zh
    num_bytes: 59168
    num_examples: 500
  - name: ta
    num_bytes: 94160
    num_examples: 500
  - name: th
    num_bytes: 181602
    num_examples: 500
  - name: tr
    num_bytes: 58138
    num_examples: 500
  - name: vi
    num_bytes: 62771
    num_examples: 500
  download_size: 1315174
  dataset_size: 822967
- config_name: open_llama_13b
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 56288
    num_examples: 500
  - name: ht
    num_bytes: 54954
    num_examples: 500
  - name: it
    num_bytes: 59628
    num_examples: 500
  - name: id
    num_bytes: 58167
    num_examples: 500
  - name: qu
    num_bytes: 89296
    num_examples: 500
  - name: sw
    num_bytes: 59578
    num_examples: 500
  - name: zh
    num_bytes: 58133
    num_examples: 500
  - name: ta
    num_bytes: 94160
    num_examples: 500
  - name: th
    num_bytes: 186125
    num_examples: 500
  - name: tr
    num_bytes: 56290
    num_examples: 500
  - name: vi
    num_bytes: 58354
    num_examples: 500
  download_size: 1340180
  dataset_size: 830973
- config_name: open_llama_7b_v2
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 53471
    num_examples: 500
  - name: ht
    num_bytes: 55430
    num_examples: 500
  - name: it
    num_bytes: 59523
    num_examples: 500
  - name: id
    num_bytes: 57590
    num_examples: 500
  - name: qu
    num_bytes: 87887
    num_examples: 500
  - name: sw
    num_bytes: 62658
    num_examples: 500
  - name: zh
    num_bytes: 57696
    num_examples: 500
  - name: ta
    num_bytes: 94160
    num_examples: 500
  - name: th
    num_bytes: 58255
    num_examples: 500
  - name: tr
    num_bytes: 54985
    num_examples: 500
  - name: vi
    num_bytes: 57207
    num_examples: 500
  download_size: 1066611
  dataset_size: 698862
- config_name: falcon-7b
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 80694
    num_examples: 500
  - name: ht
    num_bytes: 64949
    num_examples: 500
  - name: it
    num_bytes: 60169
    num_examples: 500
  - name: id
    num_bytes: 57919
    num_examples: 500
  - name: qu
    num_bytes: 82389
    num_examples: 500
  - name: sw
    num_bytes: 68738
    num_examples: 500
  - name: zh
    num_bytes: 62816
    num_examples: 500
  - name: ta
    num_bytes: 16427
    num_examples: 500
  - name: th
    num_bytes: 155861
    num_examples: 500
  - name: tr
    num_bytes: 64322
    num_examples: 500
  - name: vi
    num_bytes: 94137
    num_examples: 500
  download_size: 1302140
  dataset_size: 808421
- config_name: xgen-7b-4k-base
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 58498
    num_examples: 500
  - name: ht
    num_bytes: 55498
    num_examples: 500
  - name: it
    num_bytes: 59696
    num_examples: 500
  - name: id
    num_bytes: 55936
    num_examples: 500
  - name: qu
    num_bytes: 80560
    num_examples: 500
  - name: sw
    num_bytes: 65035
    num_examples: 500
  - name: zh
    num_bytes: 58163
    num_examples: 500
  - name: ta
    num_bytes: 14813
    num_examples: 500
  - name: th
    num_bytes: 64876
    num_examples: 500
  - name: tr
    num_bytes: 57701
    num_examples: 500
  - name: vi
    num_bytes: 58791
    num_examples: 500
  download_size: 997295
  dataset_size: 629567
- config_name: xgen-7b-8k-base
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 57918
    num_examples: 500
  - name: ht
    num_bytes: 55553
    num_examples: 500
  - name: it
    num_bytes: 59322
    num_examples: 500
  - name: id
    num_bytes: 56829
    num_examples: 500
  - name: qu
    num_bytes: 93371
    num_examples: 500
  - name: sw
    num_bytes: 65770
    num_examples: 500
  - name: zh
    num_bytes: 57378
    num_examples: 500
  - name: ta
    num_bytes: 14813
    num_examples: 500
  - name: th
    num_bytes: 60694
    num_examples: 500
  - name: tr
    num_bytes: 56341
    num_examples: 500
  - name: vi
    num_bytes: 58305
    num_examples: 500
  download_size: 1003224
  dataset_size: 636294
- config_name: xgen-7b-8k-inst
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 57938
    num_examples: 500
  - name: ht
    num_bytes: 59577
    num_examples: 500
  - name: it
    num_bytes: 58999
    num_examples: 500
  - name: id
    num_bytes: 57198
    num_examples: 500
  - name: qu
    num_bytes: 74792
    num_examples: 500
  - name: sw
    num_bytes: 63739
    num_examples: 500
  - name: zh
    num_bytes: 58638
    num_examples: 500
  - name: ta
    num_bytes: 14813
    num_examples: 500
  - name: th
    num_bytes: 64762
    num_examples: 500
  - name: tr
    num_bytes: 58008
    num_examples: 500
  - name: vi
    num_bytes: 56758
    num_examples: 500
  download_size: 992574
  dataset_size: 625222
- config_name: polylm-1.7b
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 127291
    num_examples: 500
  - name: ht
    num_bytes: 100114
    num_examples: 500
  - name: it
    num_bytes: 70393
    num_examples: 500
  - name: id
    num_bytes: 58829
    num_examples: 500
  - name: qu
    num_bytes: 92265
    num_examples: 500
  - name: sw
    num_bytes: 88160
    num_examples: 500
  - name: zh
    num_bytes: 56896
    num_examples: 500
  - name: ta
    num_bytes: 123071
    num_examples: 500
  - name: th
    num_bytes: 67106
    num_examples: 500
  - name: tr
    num_bytes: 107151
    num_examples: 500
  - name: vi
    num_bytes: 56025
    num_examples: 500
  download_size: 1326335
  dataset_size: 947301
- config_name: polylm-13b
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 52813
    num_examples: 500
  - name: ht
    num_bytes: 57552
    num_examples: 500
  - name: it
    num_bytes: 58876
    num_examples: 500
  - name: id
    num_bytes: 58351
    num_examples: 500
  - name: qu
    num_bytes: 67767
    num_examples: 500
  - name: sw
    num_bytes: 52179
    num_examples: 500
  - name: zh
    num_bytes: 56913
    num_examples: 500
  - name: ta
    num_bytes: 151911
    num_examples: 500
  - name: th
    num_bytes: 56069
    num_examples: 500
  - name: tr
    num_bytes: 56251
    num_examples: 500
  - name: vi
    num_bytes: 56378
    num_examples: 500
  download_size: 1093006
  dataset_size: 725060
- config_name: polylm-multialpaca-13b
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 50900
    num_examples: 500
  - name: ht
    num_bytes: 55054
    num_examples: 500
  - name: it
    num_bytes: 58941
    num_examples: 500
  - name: id
    num_bytes: 58062
    num_examples: 500
  - name: qu
    num_bytes: 66646
    num_examples: 500
  - name: sw
    num_bytes: 55903
    num_examples: 500
  - name: zh
    num_bytes: 57690
    num_examples: 500
  - name: ta
    num_bytes: 159507
    num_examples: 500
  - name: th
    num_bytes: 54790
    num_examples: 500
  - name: tr
    num_bytes: 56229
    num_examples: 500
  - name: vi
    num_bytes: 56748
    num_examples: 500
  download_size: 1097212
  dataset_size: 730470
- config_name: open_llama_3b_v2
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 55145
    num_examples: 500
  - name: ht
    num_bytes: 55602
    num_examples: 500
  - name: it
    num_bytes: 59546
    num_examples: 500
  - name: id
    num_bytes: 57579
    num_examples: 500
  - name: qu
    num_bytes: 72123
    num_examples: 500
  - name: sw
    num_bytes: 62381
    num_examples: 500
  - name: zh
    num_bytes: 58425
    num_examples: 500
  - name: ta
    num_bytes: 106600
    num_examples: 500
  - name: th
    num_bytes: 64880
    num_examples: 500
  - name: tr
    num_bytes: 57858
    num_examples: 500
  - name: vi
    num_bytes: 61197
    num_examples: 500
  download_size: 1078124
  dataset_size: 711336
- config_name: Llama-2-7b-hf
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 55987
    num_examples: 500
  - name: ht
    num_bytes: 55689
    num_examples: 500
  - name: it
    num_bytes: 59478
    num_examples: 500
  - name: id
    num_bytes: 58155
    num_examples: 500
  - name: qu
    num_bytes: 64673
    num_examples: 500
  - name: sw
    num_bytes: 59586
    num_examples: 500
  - name: zh
    num_bytes: 57100
    num_examples: 500
  - name: ta
    num_bytes: 84633
    num_examples: 500
  - name: th
    num_bytes: 55732
    num_examples: 500
  - name: tr
    num_bytes: 55864
    num_examples: 500
  - name: vi
    num_bytes: 55716
    num_examples: 500
  download_size: 1029561
  dataset_size: 662613
- config_name: Llama-2-13b-hf
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 57638
    num_examples: 500
  - name: ht
    num_bytes: 58376
    num_examples: 500
  - name: it
    num_bytes: 59731
    num_examples: 500
  - name: id
    num_bytes: 57842
    num_examples: 500
  - name: qu
    num_bytes: 67524
    num_examples: 500
  - name: sw
    num_bytes: 63141
    num_examples: 500
  - name: zh
    num_bytes: 57165
    num_examples: 500
  - name: ta
    num_bytes: 68926
    num_examples: 500
  - name: th
    num_bytes: 56742
    num_examples: 500
  - name: tr
    num_bytes: 56300
    num_examples: 500
  - name: vi
    num_bytes: 56077
    num_examples: 500
  download_size: 1026046
  dataset_size: 659462
- config_name: Llama-2-7b-chat-hf
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 50593
    num_examples: 500
  - name: ht
    num_bytes: 64307
    num_examples: 500
  - name: it
    num_bytes: 25365
    num_examples: 500
  - name: id
    num_bytes: 51404
    num_examples: 500
  - name: qu
    num_bytes: 77738
    num_examples: 500
  - name: sw
    num_bytes: 64286
    num_examples: 500
  - name: zh
    num_bytes: 21421
    num_examples: 500
  - name: ta
    num_bytes: 80610
    num_examples: 500
  - name: th
    num_bytes: 66935
    num_examples: 500
  - name: tr
    num_bytes: 54474
    num_examples: 500
  - name: vi
    num_bytes: 28370
    num_examples: 500
  download_size: 952208
  dataset_size: 585503
- config_name: Llama-2-13b-chat-hf
  features:
  - name: premise
    dtype: string
  - name: choice1
    dtype: string
  - name: choice2
    dtype: string
  - name: question
    dtype: string
  - name: label
    dtype: int32
  - name: idx
    dtype: int32
  - name: changed
    dtype: bool
  splits:
  - name: et
    num_bytes: 60368
    num_examples: 500
  - name: ht
    num_bytes: 65837
    num_examples: 500
  - name: it
    num_bytes: 59658
    num_examples: 500
  - name: id
    num_bytes: 59141
    num_examples: 500
  - name: qu
    num_bytes: 80708
    num_examples: 500
  - name: sw
    num_bytes: 66850
    num_examples: 500
  - name: zh
    num_bytes: 59536
    num_examples: 500
  - name: ta
    num_bytes: 91955
    num_examples: 500
  - name: th
    num_bytes: 65147
    num_examples: 500
  - name: tr
    num_bytes: 56932
    num_examples: 500
  - name: vi
    num_bytes: 57445
    num_examples: 500
  download_size: 1090195
  dataset_size: 723577
---

# Dataset Card for XCOPA MT

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Homepage:** [https://github.com/cambridgeltl/xcopa](https://github.com/cambridgeltl/xcopa)
- **Repository:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Paper:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Point of Contact:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Size of downloaded dataset files:** 4.08 MB
- **Size of the generated dataset:** 1.02 MB
- **Total amount of disk used:** 5.10 MB

### Dataset Summary

  XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning
The Cross-lingual Choice of Plausible Alternatives dataset is a benchmark to evaluate the ability of machine learning models to transfer commonsense reasoning across
languages. The dataset is the translation and reannotation of the English COPA (Roemmele et al. 2011) and covers 11 languages from 11 families and several areas around
the globe. The dataset is challenging as it requires both the command of world knowledge and the ability to generalise to new languages. All the details about the
creation of XCOPA and the implementation of the baselines are available in the paper.

Xcopa language et

### Supported Tasks and Leaderboards

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Languages

- et
- ht
- id
- it
- qu
- sw
- ta
- th
- tr
- vi
- zh

## Dataset Structure

### Data Instances

#### et

- **Size of downloaded dataset files:** 0.37 MB
- **Size of the generated dataset:** 0.07 MB
- **Total amount of disk used:** 0.44 MB

An example of 'validation' looks as follows.
```
{
    "changed": false,
    "choice1": "Ta kallas piima kaussi.",
    "choice2": "Ta kaotas oma isu.",
    "idx": 1,
    "label": 1,
    "premise": "Tüdruk leidis oma helveste seest putuka.",
    "question": "effect"
}
```

#### ht

- **Size of downloaded dataset files:** 0.37 MB
- **Size of the generated dataset:** 0.07 MB
- **Total amount of disk used:** 0.44 MB

An example of 'validation' looks as follows.
```
{
    "changed": false,
    "choice1": "Ta kallas piima kaussi.",
    "choice2": "Ta kaotas oma isu.",
    "idx": 1,
    "label": 1,
    "premise": "Tüdruk leidis oma helveste seest putuka.",
    "question": "effect"
}
```

#### id

- **Size of downloaded dataset files:** 0.37 MB
- **Size of the generated dataset:** 0.07 MB
- **Total amount of disk used:** 0.45 MB

An example of 'validation' looks as follows.
```
{
    "changed": false,
    "choice1": "Ta kallas piima kaussi.",
    "choice2": "Ta kaotas oma isu.",
    "idx": 1,
    "label": 1,
    "premise": "Tüdruk leidis oma helveste seest putuka.",
    "question": "effect"
}
```

#### it

- **Size of downloaded dataset files:** 0.37 MB
- **Size of the generated dataset:** 0.08 MB
- **Total amount of disk used:** 0.45 MB

An example of 'validation' looks as follows.
```
{
    "changed": false,
    "choice1": "Ta kallas piima kaussi.",
    "choice2": "Ta kaotas oma isu.",
    "idx": 1,
    "label": 1,
    "premise": "Tüdruk leidis oma helveste seest putuka.",
    "question": "effect"
}
```

#### qu

- **Size of downloaded dataset files:** 0.37 MB
- **Size of the generated dataset:** 0.08 MB
- **Total amount of disk used:** 0.45 MB

An example of 'validation' looks as follows.
```
{
    "changed": false,
    "choice1": "Ta kallas piima kaussi.",
    "choice2": "Ta kaotas oma isu.",
    "idx": 1,
    "label": 1,
    "premise": "Tüdruk leidis oma helveste seest putuka.",
    "question": "effect"
}
```

### Data Fields

The data fields are the same among all splits.

#### et
- `premise`: a `string` feature.
- `choice1`: a `string` feature.
- `choice2`: a `string` feature.
- `question`: a `string` feature.
- `label`: a `int32` feature.
- `idx`: a `int32` feature.
- `changed`: a `bool` feature.

#### ht
- `premise`: a `string` feature.
- `choice1`: a `string` feature.
- `choice2`: a `string` feature.
- `question`: a `string` feature.
- `label`: a `int32` feature.
- `idx`: a `int32` feature.
- `changed`: a `bool` feature.

#### id
- `premise`: a `string` feature.
- `choice1`: a `string` feature.
- `choice2`: a `string` feature.
- `question`: a `string` feature.
- `label`: a `int32` feature.
- `idx`: a `int32` feature.
- `changed`: a `bool` feature.

#### it
- `premise`: a `string` feature.
- `choice1`: a `string` feature.
- `choice2`: a `string` feature.
- `question`: a `string` feature.
- `label`: a `int32` feature.
- `idx`: a `int32` feature.
- `changed`: a `bool` feature.

#### qu
- `premise`: a `string` feature.
- `choice1`: a `string` feature.
- `choice2`: a `string` feature.
- `question`: a `string` feature.
- `label`: a `int32` feature.
- `idx`: a `int32` feature.
- `changed`: a `bool` feature.

### Data Splits

|name|validation|test|
|----|---------:|---:|
|et  |       100| 500|
|ht  |       100| 500|
|id  |       100| 500|
|it  |       100| 500|
|qu  |       100| 500|

## Dataset Creation

### Curation Rationale

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Source Data

#### Initial Data Collection and Normalization

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

#### Who are the source language producers?

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Annotations

#### Annotation process

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

#### Who are the annotators?

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Personal and Sensitive Information

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Discussion of Biases

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Other Known Limitations

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

## Additional Information

### Dataset Curators

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Licensing Information

[Creative Commons Attribution 4.0 International (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/).

### Citation Information

```
  @article{ponti2020xcopa,
  title={{XCOPA: A} Multilingual Dataset for Causal Commonsense Reasoning},
  author={Edoardo M. Ponti, Goran Glava
{s}, Olga Majewska, Qianchu Liu, Ivan Vuli'{c} and Anna Korhonen},
  journal={arXiv preprint},
  year={2020},
  url={https://ducdauge.github.io/files/xcopa.pdf}
}

@inproceedings{roemmele2011choice,
  title={Choice of plausible alternatives: An evaluation of commonsense causal reasoning},
  author={Roemmele, Melissa and Bejan, Cosmin Adrian and Gordon, Andrew S},
  booktitle={2011 AAAI Spring Symposium Series},
  year={2011},
  url={https://people.ict.usc.edu/~gordon/publications/AAAI-SPRING11A.PDF},
}

```

### Contributions

Thanks to [@patrickvonplaten](https://github.com/patrickvonplaten), [@lewtun](https://github.com/lewtun), [@thomwolf](https://github.com/thomwolf) for adding this dataset.