task_categories:text2text-generation task_ids:parsing annotations_creators:machine-generated language_creators:machine-generated multilinguality:multilingual size_categories:1M<n<10M source_datasets:original language:as language:bn language:gu language:hi language:kn language:ml language:mr language:or language:pa language:ta language:te license:cc0-1.0 arxiv:2304.13005 region:us
    IE-SemParse is an Inter-bilingual Seq2seq Semantic parsing dataset for 11 distinct Indian languages
@misc{aggarwal2023evaluating,
      title={Evaluating Inter-Bilingual Semantic Parsing for Indian Languages}, 
      author={Divyanshu Aggarwal and Vivek Gupta and Anoop Kunchukuttan},
      year={2023},
      eprint={2304.13005},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
---
annotations_creators:
- machine-generated
language_creators:
- machine-generated
language:
- as
- bn
- gu
- hi
- kn
- ml
- mr
- or
- pa
- ta
- te
license:
- cc0-1.0
multilinguality:
- multilingual
pretty_name: IE-SemParse
size_categories:
- 1M<n<10M
source_datasets:
- original
task_categories:
-  text2text-generation
task_ids:
- parsing
---

# Dataset Card for "IE-SemParse"

## Table of Contents

- [Dataset Card for "IE-SemParse"](#dataset-card-for-ie-semparse)
  - [Table of Contents](#table-of-contents)
  - [Dataset Description](#dataset-description)
    - [Dataset Summary](#dataset-summary)
    - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
    - [Languages](#languages)
  - [Dataset usage](#dataset-usage)
  - [Dataset Creation](#dataset-creation)
    - [Curation Rationale](#curation-rationale)
    - [Source Data](#source-data)
      - [Initial Data Collection and Normalization](#initial-data-collection-and-normalization)
      - [Who are the source language producers?](#who-are-the-source-language-producers)
      - [Human Verification Process](#human-verification-process)
  - [Considerations for Using the Data](#considerations-for-using-the-data)
    - [Social Impact of Dataset](#social-impact-of-dataset)
    - [Discussion of Biases](#discussion-of-biases)
    - [Other Known Limitations](#other-known-limitations)
    - [Dataset Curators](#dataset-curators)
    - [Licensing Information](#licensing-information)
    - [Citation Information](#citation-information)

## Dataset Description

- **Homepage:** <https://github.com/divyanshuaggarwal/IE-SemParse>
- **Paper:** [Evaluating Inter-Bilingual Semantic Parsing for Indian Languages](https://arxiv.org/abs/2304.13005)
- **Point of Contact:** [Divyanshu Aggarwal](mailto:divyanshuggrwl@gmail.com)

### Dataset Summary

IE-SemParse is an InterBilingual Semantic Parsing Dataset for eleven major Indic languages that includes
Assamese (‘as’), Gujarat (‘gu’), Kannada (‘kn’),
Malayalam (‘ml’), Marathi (‘mr’), Odia (‘or’),
Punjabi (‘pa’), Tamil (‘ta’), Telugu (‘te’), Hindi
(‘hi’), and Bengali (‘bn’).

### Supported Tasks and Leaderboards

**Tasks:** Inter-Bilingual Semantic Parsing

**Leaderboards:** Currently there is no Leaderboard for this dataset.

### Languages

- `Assamese (as)`
- `Bengali (bn)`
- `Gujarati (gu)`
- `Kannada (kn)`
- `Hindi (hi)`
- `Malayalam (ml)`
- `Marathi (mr)`
- `Oriya (or)`
- `Punjabi (pa)`
- `Tamil (ta)`
- `Telugu (te)`

...



<!-- Below is the dataset split given for `hi` dataset.

```python
DatasetDict({
    train: Dataset({
        features: ['utterance', 'logical form', 'intent'],
        num_rows: 36000
    })
    test: Dataset({
        features: ['utterance', 'logical form', 'intent'],
        num_rows: 3000
    })
    validation: Dataset({
        features: ['utterance', 'logical form', 'intent'],
        num_rows: 1500
    })
})

``` -->


## Dataset usage

Code snippet for using the dataset using datasets library.

```python
from datasets import load_dataset

dataset = load_dataset("Divyanshu/IE_SemParse")
```

## Dataset Creation

Machine translation of 3 multilingual semantic Parsing datasets english dataset to 11 listed Indic Languages.

### Curation Rationale

[More information needed]

### Source Data

[mTOP dataset](https://aclanthology.org/2021.eacl-main.257/)

[multilingualTOP dataset](https://github.com/awslabs/multilingual-top)

[multi-ATIS++ dataset](https://paperswithcode.com/paper/end-to-end-slot-alignment-and-recognition-for)

#### Initial Data Collection and Normalization

[Detailed in the paper](https://arxiv.org/abs/2304.13005)

#### Who are the source language producers?

[Detailed in the paper](https://arxiv.org/abs/2304.13005)

#### Human Verification Process

[Detailed in the paper](https://arxiv.org/abs/2304.13005)

## Considerations for Using the Data

### Social Impact of Dataset

[Detailed in the paper](https://arxiv.org/abs/2304.13005)

### Discussion of Biases

[Detailed in the paper](https://arxiv.org/abs/2304.13005)

### Other Known Limitations

[Detailed in the paper](https://arxiv.org/abs/2304.13005)

### Dataset Curators

Divyanshu Aggarwal, Vivek Gupta, Anoop Kunchukuttan

### Licensing Information

Contents of this repository are restricted to only non-commercial research purposes under the [Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0)](https://creativecommons.org/licenses/by-nc/4.0/). Copyright of the dataset contents belongs to the original copyright holders.

### Citation Information

If you use any of the datasets, models or code modules, please cite the following paper:

```
@misc{aggarwal2023evaluating,
      title={Evaluating Inter-Bilingual Semantic Parsing for Indian Languages}, 
      author={Divyanshu Aggarwal and Vivek Gupta and Anoop Kunchukuttan},
      year={2023},
      eprint={2304.13005},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```

<!-- ### Contributions -->
