task_categories:summarization size_categories:100K<n<1M language:th license:cc-by-4.0 region:us
---
dataset_info:
  features:
  - name: article
    dtype: string
  - name: highlights
    dtype: string
  - name: id
    dtype: string
  splits:
  - name: train
    num_bytes: 942933950
    num_examples: 287113
  - name: validation
    num_bytes: 45473576
    num_examples: 13368
  - name: test
    num_bytes: 38498988
    num_examples: 11490
  download_size: 401965818
  dataset_size: 1026906514
license: cc-by-4.0
task_categories:
- summarization
language:
- th
size_categories:
- 100K<n<1M
---

# Dataset Card for "cnn_dailymail_thai 🇹🇭"

- This dataset is based on [CNN/DailyMail](https://huggingface.co/datasets/cnn_dailymail) v3.0.0.
- The documents and summaries were translated from English (as in the original XSum) to Thai using Meta's [NLLB-200-3.3B](https://huggingface.co/facebook/nllb-200-3.3B). 
  - hyperparameters: max_length=(doc=1024, sum=512), num_beams=4, no_repeat_ngram_size=6, repetition_penalty=1.2
- GitHub repo: https://github.com/potsawee/thaisum_data
- The dataset is intended for Thai Summarization, e.g., ThaiNews -> ThaiSummary.
- This dataset can be cross-linked with the original (English) CNNDMM using the `id` attribute
- Note that we observe the input length is noticeably shorter than the original English version (stats to be included)
- The quality of translation appears quite poor. There will be an update version with a better translation system.
- 
### Data Fields

- `id`: a string containing the heximal formated SHA1 hash of the url where the story was retrieved from.
- `article`: a string containing the *translated* body of the news article 
- `highlights`: a string containing the *translated* highlight of the article.


## Data Structure

```
{
    "id": "42c027e4ff9730fbb3de84c1af0d2c506e41c3e4",
    "article": "news article in Thai",
    "highlights": "summary in Thai"
}
```

### Data Splits
train/validation/test = 287113/13368/11490