multilinguality:monolingual language:en license:apache-2.0 region:us
The SciTail dataset is an entailment dataset created from multiple-choice science exams and
web sentences. Each question and the correct answer choice are converted into an assertive
statement to form the hypothesis. We use information retrieval to obtain relevant text from
a large text corpus of web sentences, and use these sentences as a premise P. We crowdsource
the annotation of such premise-hypothesis pair as supports (entails) or not (neutral), in order
to create the SciTail dataset. The dataset contains 27,026 examples with 10,101 examples with
entails label and 16,925 examples with neutral label.
@article{
    Khot_Sabharwal_Clark_2018,
    title={SciTaiL: A Textual Entailment Dataset from Science Question Answering},
    volume={32},
    url={https://ojs.aaai.org/index.php/AAAI/article/view/12022}, DOI={10.1609/aaai.v32i1.12022},
    abstractNote={ &lt;p&gt; We present a new dataset and model for textual entailment, derived from treating multiple-choice question-answering as an entailment problem. SciTail is the first entailment set that is created solely from natural sentences that already exist independently ``in the wild’’ rather than sentences authored specifically for the entailment task. Different from existing entailment datasets, we create hypotheses from science questions and the corresponding answer candidates, and premises from relevant web sentences retrieved from a large corpus. These sentences are often linguistically challenging. This, combined with the high lexical similarity of premise and hypothesis for both entailed and non-entailed pairs, makes this new entailment task particularly difficult. The resulting challenge is evidenced by state-of-the-art textual entailment systems achieving mediocre performance on SciTail, especially in comparison to a simple majority class baseline. As a step forward, we demonstrate that one can improve accuracy on SciTail by 5% using a new neural model that exploits linguistic structure. &lt;/p&gt; },
    number={1},
    journal={Proceedings of the AAAI Conference on Artificial Intelligence},
    author={Khot, Tushar and Sabharwal, Ashish and Clark, Peter},
    year={2018},
    month={Apr.}
}
---
language: 
  - en
bigbio_language:
  - English
license: apache-2.0
bigbio_license_shortname: APACHE_2p0
multilinguality: monolingual
pretty_name: SciTail
homepage: https://allenai.org/data/scitail
bigbio_pubmed: false
bigbio_public: true
bigbio_tasks:
  - TEXTUAL_ENTAILMENT
paperswithcode_id: scitail
---


# Dataset Card for SciTail

## Dataset Description

- **Homepage:** https://allenai.org/data/scitail
- **Pubmed:** False
- **Public:** True
- **Tasks:** TE


The SciTail dataset is an entailment dataset created from multiple-choice science exams and web sentences. Each question and the correct answer choice are converted into an assertive statement to form the hypothesis. We use information retrieval to obtain relevant text from a large text corpus of web sentences, and use these sentences as a premise P. We crowd source the annotation of such premise-hypothesis pair as supports (entails) or not (neutral), in order to create the SciTail dataset. The dataset contains 27,026 examples with 10,101 examples with entails label and 16,925 examples with neutral label.


## Citation Information

```
@inproceedings{scitail,
    author = {Tushar Khot and Ashish Sabharwal and Peter Clark},
    booktitle = {AAAI}
    title = {SciTail: A Textual Entailment Dataset from Science Question Answering},
    year = {2018}
```
