region:us
The AI4Bharat-IndicNLP dataset is an ongoing effort to create a collection of large-scale, 
general-domain corpora for Indian languages. Currently, it contains 2.7 billion words for 10 Indian languages from two language families. 
We share pre-trained word embeddings trained on these corpora.
We create news article category classification datasets for 9 languages to evaluate the embeddings.
We evaluate the IndicNLP embeddings on multiple evaluation tasks.
@article{kunchukuttan2020indicnlpcorpus,
    title={AI4Bharat-IndicNLP Corpus: Monolingual Corpora and Word Embeddings for Indic Languages},
    author={Anoop Kunchukuttan and Divyanshu Kakwani and Satish Golla and Gokul N.C. and Avik Bhattacharyya and Mitesh M. Khapra and Pratyush Kumar},
    year={2020},
    journal={arXiv preprint arXiv:2005.00085},
}
## IndicNLP News Article Classification Dataset

We used the IndicNLP text corpora to create classification datasets comprising news articles and their categories for 9 languages. The dataset is balanced across classes.  The following table contains the statistics of our dataset:

| Language  | Classes                                     | Articles per Class |
| --------- | ------------------------------------------- | ------------------ |
| Bengali   | entertainment, sports                       | 7K                 |
| Gujarati  | business, entertainment, sports             | 680                |
| Kannada   | entertainment, lifestyle, sports            | 10K                |
| Malayalam | business, entertainment, sports, technology | 1.5K               |
| Marathi   | entertainment, lifestyle, sports            | 1.5K               |
| Oriya     | business, crime, entertainment, sports      | 7.5K               |
| Punjabi   | business, entertainment, sports, politics   | 780                |
| Tamil     | entertainment, politics, sport              | 3.9K               |
| Telugu    | entertainment, business, sports             | 8K                 |

## Citing

If you are using any of the resources, please cite the following article: 

```
@article{kunchukuttan2020indicnlpcorpus,
    title={AI4Bharat-IndicNLP Corpus: Monolingual Corpora and Word Embeddings for Indic Languages},
    author={Anoop Kunchukuttan and Divyanshu Kakwani and Satish Golla and Gokul N.C. and Avik Bhattacharyya and Mitesh M. Khapra and Pratyush Kumar},
    year={2020},
    journal={arXiv preprint arXiv:2005.00085},
}
```
