task_categories:question-answering task_categories:text-generation language:en license:apache-2.0 education children education region:us
FairytaleQA dataset, an open-source dataset focusing on comprehension of narratives, targeting students from kindergarten to eighth grade. The FairytaleQA dataset is annotated by education experts based on an evidence-based theoretical framework. It consists of 10,580 explicit and implicit questions derived from 278 children-friendly stories, covering seven types of narrative elements or relations.
@inproceedings{xu-etal-2022-fantastic,
    title = "Fantastic Questions and Where to Find Them: {F}airytale{QA} {--} An Authentic Dataset for Narrative Comprehension",
    author = "Xu, Ying  and
      Wang, Dakuo  and
      Yu, Mo  and
      Ritchie, Daniel  and
      Yao, Bingsheng  and
      Wu, Tongshuang  and
      Zhang, Zheng  and
      Li, Toby  and
      Bradford, Nora  and
      Sun, Branda  and
      Hoang, Tran  and
      Sang, Yisi  and
      Hou, Yufang  and
      Ma, Xiaojuan  and
      Yang, Diyi  and
      Peng, Nanyun  and
      Yu, Zhou  and
      Warschauer, Mark",
    booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.acl-long.34",
    doi = "10.18653/v1/2022.acl-long.34",
    pages = "447--460",
    abstract = "Question answering (QA) is a fundamental means to facilitate assessment and training of narrative comprehension skills for both machines and young children, yet there is scarcity of high-quality QA datasets carefully designed to serve this purpose. In particular, existing datasets rarely distinguish fine-grained reading skills, such as the understanding of varying narrative elements. Drawing on the reading education research, we introduce FairytaleQA, a dataset focusing on narrative comprehension of kindergarten to eighth-grade students. Generated by educational experts based on an evidence-based theoretical framework, FairytaleQA consists of 10,580 explicit and implicit questions derived from 278 children-friendly stories, covering seven types of narrative elements or relations. Our dataset is valuable in two folds: First, we ran existing QA models on our dataset and confirmed that this annotation helps assess models{'} fine-grained learning skills. Second, the dataset supports question generation (QG) task in the education domain. Through benchmarking with QG models, we show that the QG model trained on FairytaleQA is capable of asking high-quality and more diverse questions.",
}
---
license: apache-2.0
task_categories:
- question-answering
- text-generation
language:
- en
tags:
- education
- children education
---

# Dataset Card for FairytaleQA

## Dataset Description

- **Homepage:** 
- **Repository:**
 https://github.com/uci-soe/FairytaleQAData
 https://github.com/WorkInTheDark/FairytaleQA_Dataset
- **Paper:**
 https://aclanthology.org/2022.acl-long.34/
- **Leaderboard:** 
- **Point of Contact:** 

### Dataset Summary

This is the repository for the FairytaleQA dataset, an open-source dataset focusing on comprehension of narratives, targeting students from kindergarten to eighth grade. The FairytaleQA dataset is annotated by education experts based on an evidence-based theoretical framework. It consists of 10,580 explicit and implicit questions derived from 278 children-friendly stories, covering seven types of narrative elements or relations.

### Supported Tasks and Leaderboards

Question-Answering, Question-Generation, Question-Answer Pair Generation

### Languages

English

## Dataset Structure


### Data Instances

An example of "train" looks as follows:

```

{
  'story_name': 'three-dogs',
  'story_section': 'once upon a time there was a king who went forth into the world and 
                      ... ...
                    guards to watch over the little princess so that she would not get out under the open sky .',
  'question': 'why was there great rejoicing in the city and throughout the country ?',
  'answer1': 'the people wished their king all that was good .',
  'answer2': '',
  'local-or-sum': 'local',
  'attribute': 'causal relationship',
  'ex-or-im': 'explicit',
  'ex-or-im2': '',
}

```

### Data Fields

 - **'story_name'**: story name
 - **'story_section'**: story section related to the QA-pair
 - **'question'**: the question content
 - **'answer1'**: the 1st answer  (available in all splits)
 - **'answer2'**: the 2nd answer by another annotator (only available in test / val splits)
 - **'local-or-sum'**: 'local' denotes the question is related to only one story section, while 'summary' denotes the question is related to multiple story sections
 - **'attribute'**: categorized by education experts into seven narrative elements: character / setting / action / feeling / causal relationship / outcome resolution, detailed definition is described in the paper
 - **'ex-or-im'**: 'explicit' denotes the answer can be found in the story content, while 'implicit' denotes the answer require high-level summarization
 - **'ex-or-im2'**: similar to 'ex-or-im', but annotated by another annotator (only available in storys in test / val splits)



### Data Splits

- train split: 232 books with 8548 QA-pairs
- val split: 23 books with 1025 QA-pairs
- test split: 23 books with 1007 QA-pairs

## Dataset Creation

### Curation Rationale

[More Information Needed]

### Source Data

#### Initial Data Collection and Normalization

[More Information Needed]

#### Who are the source language producers?

[More Information Needed]

### Annotations

#### Annotation process

[More Information Needed]

#### Who are the annotators?

[More Information Needed]

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

[More Information Needed]

### Citation Information

Our Dataset Paper is accepted to ACL 2022, you may cite:
```
@inproceedings{xu2022fairytaleqa,
    author={Xu, Ying and Wang, Dakuo and Yu, Mo and Ritchie, Daniel and Yao, Bingsheng and Wu, Tongshuang and Zhang, Zheng and Li, Toby Jia-Jun and Bradford, Nora and Sun, Branda and Hoang, Tran Bao and Sang, Yisi and Hou, Yufang and Ma, Xiaojuan and Yang, Diyi and Peng, Nanyun and Yu, Zhou and Warschauer, Mark},
    title = {Fantastic Questions and Where to Find Them: Fairytale{QA} -- An Authentic Dataset for Narrative Comprehension},
    publisher = {Association for Computational Linguistics},
    year = {2022}
}
```

### Contributions

[More Information Needed]