language:ml license:cc-by-sa-4.0 region:us
---
language: ml
license: cc-by-sa-4.0
extra_gated_prompt: 'By accessing this dataset, you agree to abide by the BigScience
  Ethical Charter. The charter can be found at:

  https://hf.co/spaces/bigscience/ethical-charter'
extra_gated_fields:
  I have read and agree to abide by the BigScience Ethical Charter: checkbox
---
ROOTS Subset: roots_indic-ml_pib

# pib

- Dataset uid: `pib`

### Description

Sentence aligned parallel corpus between 11 Indian Languages, crawled and extracted from the press information bureau
website.


### Homepage

- https://huggingface.co/datasets/pib
- http://preon.iiit.ac.in/~jerin/bhasha/

### Licensing

Creative Commons Attribution-ShareAlike 4.0 International

### Speaker Locations



### Sizes

- 0.0609 % of total
- 0.6301 % of indic-hi
- 3.2610 % of indic-ur
- 0.6029 % of indic-ta
- 3.0834 % of indic-or
- 1.9757 % of indic-mr
- 0.2181 % of indic-bn
- 1.8901 % of indic-pa
- 1.5457 % of indic-gu
- 0.4695 % of indic-ml
- 0.5767 % of indic-te

### BigScience processing steps

#### Filters applied to: indic-hi

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-ur

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-ta

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-or

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs

#### Filters applied to: indic-mr

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-bn

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-pa

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-gu

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-ml

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

#### Filters applied to: indic-te

- dedup_document
- dedup_template_soft
- filter_remove_empty_docs
- filter_small_docs_bytes_300

