task_categories:text-generation size_categories:10M<n<100M language:vi LM region:us
---
dataset_info:
  features:
  - name: text
    dtype: string
  splits:
  - name: train
    num_bytes: 77360702833
    num_examples: 23891116
  - name: validation
    num_bytes: 4064634081
    num_examples: 1257428
  download_size: 2126869688
  dataset_size: 81425336914
task_categories:
- text-generation
language:
- vi
size_categories:
- 10M<n<100M
tags:
- LM
---
# Dataset Card for "vi-dataset-for-pretrain"
This is a combination of multiple Vietnamese dataset for pretraining CLMs such as GPT, GPT2, etc.

The dataset consists of:
- [`vietgpt/covid_19_news_vi`](https://huggingface.co/datasets/vietgpt/covid_19_news_vi)
- [`hieunguyen1053/binhvq-news-corpus`](https://huggingface.co/datasets/hieunguyen1053/binhvq-news-corpus)
- [`oscar (unshuffled_deduplicated_vi)`](https://huggingface.co/datasets/oscar)
- [`vietgpt/wikipedia_vi`](https://huggingface.co/datasets/vietgpt/wikipedia_vi)

# Dataset info

| Splits  | N.o examples | Size |
| --- | --- | --- |
| Train  | 23,891,116 | 77.36 GB |
| Validation  | 1,257,428  | 4.06 GB |
| **Total** | **25,148,544** | **81.43 GB** |