Argumentation mining aims at automatically extracting the premises-claim discourse structures in natural language texts. There is a great demand for argumentation corpora for customer reviews. However, due to the controversial nature of the argumentation annotation task, there exist very few large-scale argumentation corpora for customer reviews. In this work, we novelly use the crowdsourcing technique to collect argumentation annotations in Chinese hotel reviews. As the first Chinese argumentation dataset, our corpus includes 4814 argument component annotations and 411 argument relation annotations, and its annotations qualities are comparable to some widely used argumentation corpora in other languages.
Crowdsourcing has been used to annotate discourse structures. @cite_9 designed a two-stage crowdsourcing mechanism to annotate two levels of discourse relations in Japanese texts crawled from multiple online genres. In their work, discourse relations include contrast, concession, cause-effect, etc., and these relations closely resemble some relations in argumentation structures (e.g. the attack relation between arguments can be viewed as contrast, and the premise-claim relation is closely related to the cause-effect relation). They did not report the IRA of the crowdsourcing workers, but instead, testified the quality of their discourse corpus by training a discourse parser on their corpus. Results suggested that the quality of their corpus is comparable to the state-of-the-art English discourse corpus, indicating that crowdsourcing can be reliably used for annotating relations between clauses.
@cite_9
2110737727
We present the second version of the Penn Discourse Treebank, PDTB-2.0, describing its lexically-grounded annotations of discourse relations and their two abstract object arguments over the 1 million word Wall Street Journal corpus. We describe all aspects of the annotation, including (a) the argument structure of discourse relations, (b) the sense annotation of the relations, and (c) the attribution of discourse relations and each of their arguments. We list the differences between PDTB-1.0 and PDTB-2.0. We present representative statistics for several aspects of the annotation in the corpus.