license:cc-by-4.0 region:us
---
license: cc-by-4.0
---

# Dataset Card for Ehn-Bible-BBC-GPT3.5

## Dataset Description

- **Repository:** https://huggingface.co/datasets/NITHUB-AI/Ehn-bible-bbc-gpt3.5/
- **Paper:** To be added
- **Point of Contact:** fortuneadekogbe@gmail.com

### Dataset Summary

This dataset card contains parallel Nigerian Pidgin and English sentences split into three files, namely: `train.csv`, `valid.csv` and `test.csv`. 
The original data was split in the ratio of 8:1:1 to obtain these files.

### Supported Tasks and Leaderboards

- Language Translation
- Language Identification

### Languages

- English
- Nigerian Pidgin

## Dataset Structure

### Data Instances

| english                                                                                                                                                               | pidgin                                                                                                                                                                           |
|------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| " Do you want to kill me as you did the Egyptian yesterday?’ "                                                                                                         | " Hope yu nor won kill mi di way yu kill dat Egypt man yestiday?’ "                                                                                                              |
| " “Go and cry in the hearing of Jerusalem, saying, ‘Thus says the Lord : “I remember you, The kindness of your youth, The love of your betrothal, When you went after Me in the wilderness, In a land not sown." | " “Go tell evribody for Jerusalem sey: ‘God sey, “I remember as una dey faithful wen una dey yong. Una bin love mi well-well like woman wey just marry; una follow mi waka for wildaness and for dry land. " |
| " Therefore I take pleasure in infirmities, in reproaches, in needs, in persecutions, in distresses, for Christ’s sake. For when I am weak, then I am strong."            | " I dey happy as I dey weak with all di curse, trobols, ponishment and wahala wey I dey sofa bikos of Christ, bikos anytime wey I dey weak, na dat time, I dey-dey strong. "                                 |

### Data Fields

- English: contains sentences in the English language
- Pidgin: contains corresponding sentences in Nigerian Pidgin language

### Data Splits

- train (60%)
- validation (20%)
- test (20%)

## Dataset Creation

This section details the process involved in creating this Data.

### Curation Rationale

The data was curated from the context of the Bible, which proved to be the largest available source of English-Nigerian Pidgin parallel sentences.
For the English sentences, The Message translation of the Bible was used because it presented the most modern form of English.


### Source Data

#### Initial Data Collection and Normalization

- The data was scraped using BeautifulSoup in Python and stored in a MongoDB database
- The Bible-sourced data was split into samples by verses because that was the easiest way to retain context between parallel sentences. Primarily because sentences in English and Nigerian Pidgin were not perfect matches.

#### Who are the source language producers?

- [YouVersion PCM Bible](https://www.bible.com/bible/2516/GEN.1.PCM)
- [YouVersion Message Translation Bible](https://www.bible.com/bible/97/GEN.1.MSG)

### Personal and Sensitive Information

No additional effort was taken to remove sensitive information aside from what is present in the Bible.

## Considerations for Using the Data

### Social Impact of Dataset

This data makes it easier for Engineers to build language tools that work for a less literate but digitally connected Nigerian audience.

### Discussion of Biases

The data is primarily focused on Biblical texts in modern English. This limits the versatility of the data and the systems built on it.
 
### Other Known Limitations

- The data does not contain other versions of Pidgin, like Warri Pidgin or Pidgin from other African nations.
- The data contains a limited number of contexts, mostly from a religious angle.

## Additional Information

### Dataset Curators

- [Fortune Adekogbe](https://www.linkedin.com/in/fortune-adekogbe)
- [Joseph Olaide](https://ng.linkedin.com/in/josepholaide)

### Citation Information

- [Domot - BBC News Pidgin](https://www.bbc.com/pidgin/)
  
### Contributions

We welcome contributions from individuals who understand Nigerian Pidgin to help scale up our manual data translation efforts. Motivated developers interested in building interfaces for this are also welcome.