task_categories:translation size_categories:10K<n<100K language:en language:rw license:cc-by-2.0 region:us
---
license: cc-by-2.0
task_categories:
- translation
language:
- en
- rw
size_categories:
- 10K<n<100K
---


## Dataset Description

This dataset was created in an effort to create a machine translation model for English-to-Kinyarwanda translation and vice-versa in a tourism-geared context.
- **Repository:**[link](https://github.com/Digital-Umuganda/twb_nllb_project_tourism_education) to the GitHub repository containing the code for training the model on this data, and the code for the collection of the monolingual data.
- **Data Format:** TSV
- **Data Source:** web scraping, manual annotation
- **Model:** huggingface [model link](mbazaNLP/Nllb_finetuned_tourism_en_kin).



### Data Instances

```
25375	49363	21210	Bird watching is best in June, so save your money on that during the other months, birds ar everywhere anyway if you are observant and patient.	Kureba inyoni ni byiza cyane muri Kamena, bityo rero ujye uzigama amafaranga yawe mu gihe cy'amezi yindi, inyoni ziba hose uko byagenda kose niba witonze kandi wihanganye.	2023-05-15 18:08:54	19.0	1	3	tourism	trip_advisor	125-195
```

### Data Fields

- id
- source_id
- source
- phrase
- timestamp
- user_id
- validation_state
- validation_score
- domain
- source_files
- str_ranges

### Data Splits

- **Training Data:** 25374
- **Validation Data:** 2508
- **Test Data:** 1086

## Data Preprocessing

- **Data Splitting:** To create a test set; all data sources are equally represented in terms of the number of sentences contributed to the test dataset. In terms of sentence length, the test set distribution is similar to the sentence length distribution of the whole dataset. After picking the test set, from the remaining data the train and validation data are split using sklearn's [train_test_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html).

## Data Collection

- **Data Collection Process:** The monolingual source sentences were obtained through web-scraping of several websites, and contain both Kinyarwanda and English sentences. 

- **Data Sources:**
  -  Trip_advisor reviews on hotels and tourist attractions in Rwanda.
  -  Inyamibwa historical data.
  -  Igihe tourism news.
  -  Tourism scenarios dialogue generated by GPT-3.5.
  -  Booking.com Rwandan hotel reviews.
  -  Rwanda's wiki_travel page.



## Dataset Creation

After collecting the monolingual dataset, human translators were employed to produce translations for the collected sentences. To ensure quality, each sentence was translated more than once, and each generated translation was assigned **validation_score** that was used to pick the best translation. The test dataset was further revised to remove or correct sentences with faulty translations.



