task_categories:text-classification task_ids:semantic-similarity-classification annotations_creators:expert-generated language_creators:found multilinguality:multilingual size_categories:10K<n<100K source_datasets:original language:en language:bg language:zh language:hr language:da language:nl language:et language:fa language:ja language:ko language:it language:fr language:de license:cc-by-nc-4.0 region:us
A system's task on any of the XL-WiC datasets is to identify the intended meaning of a word in a context of a given language. XL-WiC is framed as a binary classification task. Each instance in XL-WiC has a target word w, either a verb or a noun, for which two contexts are provided. Each of these contexts triggers a specific meaning of w. The task is to identify if the occurrences of w in the two contexts correspond to the same meaning or not.

XL-WiC provides dev and test sets in the following 12 languages:

Bulgarian (BG)
Danish (DA)
German (DE)
Estonian (ET)
Farsi (FA)
French (FR)
Croatian (HR)
Italian (IT)
Japanese (JA)
Korean (KO)
Dutch (NL)
Chinese (ZH)
and training sets in the following 3 languages:

German (DE)
French (FR)
Italian (IT)
@inproceedings{raganato-etal-2020-xl-wic,
  title={XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization},
  author={Raganato, Alessandro and Pasini, Tommaso and Camacho-Collados, Jose and Pilehvar, Mohammad Taher},
  booktitle={Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  pages={7193--7206},
  year={2020}
}
---
annotations_creators:
- expert-generated
extended:
- original
language_creators:
- found
language:
- en
- bg
- zh
- hr
- da
- nl
- et
- fa
- ja
- ko
- it
- fr
- de
license:
- cc-by-nc-4.0
multilinguality:
- multilingual
size_categories:
- 10K<n<100K
source_datasets:
- original
task_categories:
- text-classification
task_ids:
- semantic-similarity-classification
---

# XL-WiC
Huggingface dataset for the XL-WiC paper [https://www.aclweb.org/anthology/2020.emnlp-main.584.pdf](https://www.aclweb.org/anthology/2020.emnlp-main.584.pdf).
Please refer to the official [website](https://pilehvar.github.io/xlwic/) for more information.


## Configurations
When loading one of the XL-WSD datasets one has to specify the training language and the target language (on which dev and test will be performed). 
Please refer to [Languages](#languages) section to see in which languages training data is available.
For example, we can load the dataset having English as training language and Italian as target language as follows:
```python
from datasets import load_dataset
dataset = load_dataset('pasinit/xlwic', 'en_it')
```

## Languages
**Training data**
- en (English)
- fr (French)
- de (German)
- it (Italian)

**Dev & Test data**
- fr (French)
- de (German)
- it (Italian)
- bg (Bulgarian)
- zh (Chinese)
- hr (Croatian)
- da (Danish)
- nl (Dutch)
- et (Estonian)
- fa (Farsi)
- ja (Japanesse)
- ko (Korean)
