language:mr license:cc-by-nc-4.0 region:us
---
language: mr
license: cc-by-nc-4.0
extra_gated_prompt: 'By accessing this dataset, you agree to abide by the BigScience
  Ethical Charter. The charter can be found at:

  https://hf.co/spaces/bigscience/ethical-charter'
extra_gated_fields:
  I have read and agree to abide by the BigScience Ethical Charter: checkbox
---
ROOTS Subset: roots_indic-mr_indic_nlp_corpus

# Indic NLP Corpus

- Dataset uid: `indic_nlp_corpus`

### Description

The IndicNLP corpus is a largescale, general-domain corpus containing 2.7 billion words for 10 Indian languages from two language families. s (IndoAryan branch and Dravidian). Each language has at least 100 million words (except Oriya).

### Homepage

https://github.com/AI4Bharat/indicnlp_corpus#publicly-available-classification-datasets

### Licensing

- non-commercial use
- cc-by-nc-sa-4.0: Creative Commons Attribution Non Commercial Share Alike 4.0 International


### Speaker Locations

- Southern Asia
- India


### Sizes

- 3.4019 % of total
- 44.4368 % of indic-hi
- 64.2943 % of indic-ta
- 70.5374 % of indic-ml
- 54.2394 % of indic-te
- 55.9105 % of indic-kn
- 61.6111 % of indic-mr
- 67.2242 % of indic-pa
- 68.1470 % of indic-or
- 64.3879 % of indic-gu
- 4.1495 % of indic-bn

### BigScience processing steps

#### Filters applied to: indic-hi

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-ta

- dedup_document
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-ml

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-te

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-kn

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-mr

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-pa

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-or

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs

#### Filters applied to: indic-gu

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-bn

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

