language:en language:fr language:ja language:pt language:ru language:zh license:cc-by-4.0 arxiv:2211.01786 arxiv:2106.12066 region:us
A multilingual collection of Winograd Schemas in six languages that can be used for evaluation of cross-lingual commonsense reasoning capabilities.
@misc{muennighoff2022crosslingual,
      title={Crosslingual Generalization through Multitask Finetuning}, 
      author={Niklas Muennighoff and Thomas Wang and Lintang Sutawika and Adam Roberts and Stella Biderman and Teven Le Scao and M Saiful Bari and Sheng Shen and Zheng-Xin Yong and Hailey Schoelkopf and Xiangru Tang and Dragomir Radev and Alham Fikri Aji and Khalid Almubarak and Samuel Albanie and Zaid Alyafeai and Albert Webson and Edward Raff and Colin Raffel},
      year={2022},
      eprint={2211.01786},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

@misc{tikhonov2021heads,
    title={It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoning},
    author={Alexey Tikhonov and Max Ryabinin},
    year={2021},
    eprint={2106.12066},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
---
language:
- en
- fr
- ja
- pt
- ru
- zh
license: cc-by-4.0
---

## XWinograd

Multilingual winograd schema challenge as used in [Crosslingual Generalization through Multitask Finetuning](https://arxiv.org/abs/2211.01786).

### Languages & Samples

- "en": 2325
- "fr": 83
- "jp": 959
- "pt": 263 
- "ru": 315
- "zh": 504

### Dataset creation

The Winograd schema challenges in this dataset combine winograd schemas from the XWinograd dataset introduced in Tikhonov et al and as it only contains 16 Chinese schemas, we add 488 Chinese schemas from `clue/cluewsc2020`.

If you only want the original xwinograd chinese schemas only, do:

`load_dataset("Muennighoff/xwinograd", "zh")["test"][0][:16]`

## Additional Information

### Citation Information

```bibtex
@misc{muennighoff2022crosslingual,
      title={Crosslingual Generalization through Multitask Finetuning}, 
      author={Niklas Muennighoff and Thomas Wang and Lintang Sutawika and Adam Roberts and Stella Biderman and Teven Le Scao and M Saiful Bari and Sheng Shen and Zheng-Xin Yong and Hailey Schoelkopf and Xiangru Tang and Dragomir Radev and Alham Fikri Aji and Khalid Almubarak and Samuel Albanie and Zaid Alyafeai and Albert Webson and Edward Raff and Colin Raffel},
      year={2022},
      eprint={2211.01786},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```

```bibtex
@misc{tikhonov2021heads,
    title={It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoning},
    author={Alexey Tikhonov and Max Ryabinin},
    year={2021},
    eprint={2106.12066},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

### License

Like the original [English winograd schema challenge](https://cs.nyu.edu/~davise/papers/WinogradSchemas/WS.html), this dataset is licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). I.e. you can use it for commercial purposes etc. :)


### Contributions

Thanks to Jordan Clive, @yongzx & @khalidalt for support on adding Chinese.

