region:us
# WMT 2016 Romanian-English Translation Dataset

The original dataset can be downloaded from [here](https://github.com/nyu-dl/dl4mt-nonauto)

You can create this dataset by simply run:

```commandline
git clone https://huggingface.co/datasets/shijli/wmt16-roen
cd wmt16-roen/data
bash prepare-wmt16.sh
```

`binarized.dist.ro-en.zip` and `binarized.dist.en-ro.zip` are distilled datasets generated by a transformer base model.
It can be built by running:

```commandline
bash prepare-wmt16-distill.sh /path/to/fairseq/model source-lang target-lang
```

To build this dataset, you need to create `binarized.zip` first. Note that the distilled dataset only uses
model-generated
target sentences, which means that different translation directions result in different datasets. Therefore, you need to
specify `source-lang` and `target-lang` explicitly. Also, you need to replace `/path/to/fairseq/model` with the path of
your pretrained model.