task_categories:translation task_categories:text-generation task_categories:text2text-generation size_categories:1M<n<10M language:en language:az license:openrail azerbaijani books azerbaijani news azerbaijani poems azerbaijani articles azerbaijani dataset region:us
Machine translation EN-AZ dataset based on Google Translate and National Library of Azerbaijan.
@InProceedings{
huggingface:dataset,
title={Machine translation EN-AZ dataset},
author={Learning Machine LLC},
year={2022}
}
---
license: openrail
task_categories:
- translation
- text-generation
- text2text-generation
language:
- en
- az
tags:
- azerbaijani books
- azerbaijani news
- azerbaijani poems
- azerbaijani articles
- azerbaijani dataset
pretty_name: English-Azerbaijani Dataset
size_categories:
- 1M<n<10M
---

# Description
Dataset used to train our mT5 based model for machine translation, extracted from various text sources of National Library of Azerbaijan: [mT5-translation-enaz](https://huggingface.co/learningmachineaz/mt5-enaz-10m) \
It has only clean texts. Wiki articles wasn't used as they contain a lot of irrelevant data.

| Key point               | Info |
|-------------------------|---------|
| Rows | ~10mil. EN-AZ sentence pairs    |
| Size | 975M (zipped) / 2.8G (unzipped)    |
| Format | TSV (tab separated pairs)    |
| English | Google Translate    |
| Azerbaijani | Original cleaned text    |

## Author
Collected and prepared by [Renat Kalimulin](https://www.linkedin.com/in/rinat-kalimulin-16853358/)
