language:ind self-supervised-pretraining region:us
    Indo4B is a large-scale Indonesian self-supervised pre-training corpus
    consists of around 3.6B words, with around 250M sentences. The corpus
    covers both formal and colloquial Indonesian sentences compiled from 
    12 sources, of which two cover Indonesian colloquial language, eight
    cover formal Indonesian language, and the rest have a mixed style of
    both colloquial and formal.
    @inproceedings{wilie-etal-2020-indonlu,
        title = "{I}ndo{NLU}: Benchmark and Resources for Evaluating {I}ndonesian 
            Natural Language Understanding",
        author = "Wilie, Bryan  and
          Vincentio, Karissa  and
          Winata, Genta Indra  and
          Cahyawijaya, Samuel  and
          Li, Xiaohong  and
          Lim, Zhi Yuan  and
          Soleman, Sidik  and
          Mahendra, Rahmad  and
          Fung, Pascale  and
          Bahar, Syafri  and
          Purwarianti, Ayu",
        booktitle = "Proceedings of the 1st Conference of the Asia-Pacific Chapter of the 
                Association for Computational Linguistics and the 10th International Joint 
                Conference on Natural Language Processing",
        month = dec,
        year = "2020",
        address = "Suzhou, China",
        publisher = "Association for Computational Linguistics",
        url = "https://aclanthology.org/2020.aacl-main.85",
        pages = "843--857",
        abstract = "Although Indonesian is known to be the fourth most frequently used language 
            over the internet, the research progress on this language in natural language processing (NLP) 
            is slow-moving due to a lack of available resources. In response, we introduce the first-ever vast 
            resource for training, evaluation, and benchmarking on Indonesian natural language understanding 
            (IndoNLU) tasks. IndoNLU includes twelve tasks, ranging from single sentence classification to 
            pair-sentences sequence labeling with different levels of complexity. The datasets for the tasks 
            lie in different domains and styles to ensure task diversity. We also provide a set of Indonesian 
            pre-trained models (IndoBERT) trained from a large and clean Indonesian dataset (Indo4B) collected 
            from publicly available sources such as social media texts, blogs, news, and websites. 
            We release baseline models for all twelve tasks, as well as the framework for benchmark evaluation, 
            thus enabling everyone to benchmark their system performances.",
    }
---
tags:
- self-supervised-pretraining
language:
- ind
---

# indo4b

Indo4B is a large-scale Indonesian self-supervised pre-training corpus

    consists of around 3.6B words, with around 250M sentences. The corpus

    covers both formal and colloquial Indonesian sentences compiled from 

    12 sources, of which two cover Indonesian colloquial language, eight

    cover formal Indonesian language, and the rest have a mixed style of

    both colloquial and formal.

## Dataset Usage

Run `pip install nusacrowd` before loading the dataset through HuggingFace's `load_dataset`.

## Citation

```
@inproceedings{wilie-etal-2020-indonlu,
        title = "{I}ndo{NLU}: Benchmark and Resources for Evaluating {I}ndonesian 
            Natural Language Understanding",
        author = "Wilie, Bryan  and
          Vincentio, Karissa  and
          Winata, Genta Indra  and
          Cahyawijaya, Samuel  and
          Li, Xiaohong  and
          Lim, Zhi Yuan  and
          Soleman, Sidik  and
          Mahendra, Rahmad  and
          Fung, Pascale  and
          Bahar, Syafri  and
          Purwarianti, Ayu",
        booktitle = "Proceedings of the 1st Conference of the Asia-Pacific Chapter of the 
                Association for Computational Linguistics and the 10th International Joint 
                Conference on Natural Language Processing",
        month = dec,
        year = "2020",
        address = "Suzhou, China",
        publisher = "Association for Computational Linguistics",
        url = "https://aclanthology.org/2020.aacl-main.85",
        pages = "843--857",
        abstract = "Although Indonesian is known to be the fourth most frequently used language 
            over the internet, the research progress on this language in natural language processing (NLP) 
            is slow-moving due to a lack of available resources. In response, we introduce the first-ever vast 
            resource for training, evaluation, and benchmarking on Indonesian natural language understanding 
            (IndoNLU) tasks. IndoNLU includes twelve tasks, ranging from single sentence classification to 
            pair-sentences sequence labeling with different levels of complexity. The datasets for the tasks 
            lie in different domains and styles to ensure task diversity. We also provide a set of Indonesian 
            pre-trained models (IndoBERT) trained from a large and clean Indonesian dataset (Indo4B) collected 
            from publicly available sources such as social media texts, blogs, news, and websites. 
            We release baseline models for all twelve tasks, as well as the framework for benchmark evaluation, 
            thus enabling everyone to benchmark their system performances.",
    }
```

## License

CC0

## Homepage

[https://github.com/IndoNLP/indonlu](https://github.com/IndoNLP/indonlu)

### NusaCatalogue

For easy indexing and metadata: [https://indonlp.github.io/nusa-catalogue](https://indonlp.github.io/nusa-catalogue)