Create README.md

README.md

@@ -0,0 +1,51 @@
+---
+language: ug
+license: mit
+---
+
+# gpt2-wechsel-uyghur
+
+Model trained with WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.
+
+See the code here: https://github.com/CPJKU/wechsel
+
+And the paper here: https://arxiv.org/abs/2112.06598
+
+## Performance
+
+| Model | PPL |
+|---|---|
+| `gpt2-wechsel-sundanese` | **111.72** |
+| `gpt2` (retrained from scratch)  | 149.46 |
+
+| Model | PPL |
+|---|---|
+| `gpt2-wechsel-scottish-gaelic` | **16.43** |
+| `gpt2` (retrained from scratch)  | 19.53 |
+
+| Model | PPL |
+|---|---|
+| `gpt2-wechsel-uyghur` | **34.33** |
+| `gpt2` (retrained from scratch)  | 42.82 |
+
+| Model | PPL |
+|---|---|
+| `gpt2-wechsel-malagasy` | **14.01** |
+| `gpt2` (retrained from scratch)  | 15.93 |
+
+See our paper for details.
+
+## Citation
+
+Please cite WECHSEL as
+
+```
+@misc{minixhofer2021wechsel,
+      title={WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models}, 
+      author={Benjamin Minixhofer and Fabian Paischer and Navid Rekabsaz},
+      year={2021},
+      eprint={2112.06598},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL}
+}
+```