region:us
The dataset contains question-answer pairs to model verbal predicate-argument structure. 
The questions start with wh-words (Who, What, Where, What, etc.) and contain a verb predicate in the sentence; the answers are phrases in the sentence.
This dataset, a.k.a "QASRL-GS" (Gold Standard) or "QASRL-2020", was constructed via controlled crowdsourcing.
See the paper for details: Controlled Crowdsourcing for High-Quality QA-SRL Annotation, Roit et. al., 2020
@inproceedings{roit2020controlled,
  title={Controlled Crowdsourcing for High-Quality QA-SRL Annotation},
  author={Roit, Paul and Klein, Ayal and Stepanov, Daniela and Mamou, Jonathan and Michael, Julian and Stanovsky, Gabriel and Zettlemoyer, Luke and Dagan, Ido},
  booktitle={Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  pages={7008--7013},
  year={2020}
}
# QA-SRL 2020 (Gold Standard) 
The dataset contains question-answer pairs to model verbal predicate-argument structure. 
The questions start with wh-words (Who, What, Where, What, etc.) and contain a verb predicate in the sentence; the answers are phrases in the sentence.
This dataset, a.k.a "QASRL-GS" (Gold Standard) or "QASRL-2020", which was constructed via controlled crowdsourcing, includes high-quality QA-SRL annotations to serve as an evaluation set (dev and test) for models trained on the large-scale QA-SRL dataset (you can find it in this hub as [biu-nlp/qa_srl2018](https://huggingface.co/datasets/biu-nlp/qa_srl2018)).

See the paper for details: [Controlled Crowdsourcing for High-Quality QA-SRL Annotation, Roit et. al., 2020](https://aclanthology.org/2020.acl-main.626/).

Check out our [GitHub repository](https://github.com/plroit/qasrl-gs) to find code for evaluation. 

The dataset was annotated by selected workers from Amazon Mechanical Turk.