Argumentation Mining addresses the challenging tasks of identifying boundaries of argumentative text fragments and extracting their relationships. Fully automated solutions do not reach satisfactory accuracy due to their insufficient incorporation of semantics and domain knowledge. Therefore, experts currently rely on time-consuming manual annotations. In this paper, we present a visual analytics system that augments the manual annotation process by automatically suggesting which text fragments to annotate next. The accuracy of those suggestions is improved over time by incorporating linguistic knowledge and language modeling to learn a measure of argument similarity from user interactions. Based on a long-term collaboration with domain experts, we identify and model five high-level analysis tasks. We enable close reading and note-taking, annotation of arguments, argument reconstruction, extraction of argument relations, and exploration of argument graphs. To avoid context switches, we transition between all views through seamless morphing, visually anchoring all text- and graph-based layers. We evaluate our system with a two-stage expert user study based on a corpus of presidential debates. The results show that experts prefer our system over existing solutions due to the speedup provided by the automatic suggestions and the tight integration between text and graph views.
In recent years, several web-based interfaces have been created to support users in various text annotation tasks. For example, @cite_46 can be used for the annotation of POS tags or named entities. In this interface, annotations are made directly in the text by dragging the mouse over multiple words or clicking on a single word. employs the same interactions for text annotation. Another web-based annotation tool is called @cite_65 ; it allows annotations of named entities and their relations. @cite_49 use automatic entity extraction for annotating relationships between media streams. TimeLineCurator @cite_28 automatically extracts temporal events from unstructured text data and enables users to curate them in a visual, annotated timeline. @cite_54 have presented a collaborative text annotation framework and emphasize the importance of pre-annotation to significantly reduce annotation costs. have presented a framework that creates BRAT-compatible pre-annotations @cite_22 and discuss (dis-)advantages of pre-annotation. The initial suggestions of could be seen as pre-annotations, but are automatically updated after each interaction.
@cite_22 @cite_28 @cite_54 @cite_65 @cite_49 @cite_46
2251026067 1493490255 1784290353 2136409007
In this paper, we present a flexible approach to the efficient and exhaustive manual annotation of text documents. For this purpose, we extend WebAnno (, 2013) an open-source web-based annotation tool. 1 While it was previously limited to specific annotation layers, our extension allows adding and configuring an arbitrary number of layers through a web-based UI. These layers can be annotated separately or simultaneously, and support most types of linguistic annotations such as spans, semantic classes, dependency relations, lexical chains, and morphology. Further, we tightly integrate a generic machine learning component for automatic annotation suggestions of span annotations. In two case studies, we show that automatic annotation suggestions, combined with our split-pane UI concept, significantly reduces annotation time. Traditionally, Information Extraction (IE) has focused on satisfying precise, narrow, pre-specified requests from small homogeneous corpora (e.g., extract the location and time of seminars from a set of announcements). Shifting to a new domain requires the user to name the target relations and to manually create new extraction rules or hand-tag new training examples. This manual labor scales linearly with the number of target relations. This paper introduces Open IE (OIE), a new extraction paradigm where the system makes a single data-driven pass over its corpus and extracts a large set of relational tuples without requiring any human input. The paper also introduces TEXTRUNNER, a fully implemented, highly scalable OIE system where the tuples are assigned a probability and indexed to support efficient extraction and exploration via user queries. We report on experiments over a 9,000,000 Web page corpus that compare TEXTRUNNER with KNOWITALL, a state-of-the-art Web IE system. TEXTRUNNER achieves an error reduction of 33 on a comparable set of extractions. Furthermore, in the amount of time it takes KNOWITALL to perform extraction for a handful of pre-specified relations, TEXTRUNNER extracts a far broader set of facts reflecting orders of magnitude more relations, discovered on the fly. We report statistics on TEXTRUNNER's 11,000,000 highest probability tuples, and show that they contain over 1,000,000 concrete facts and over 6,500,000 more abstract assertions. Over the last decades, several billion Web pages have been made available on the Web. The ongoing transition from the current Web of unstructured data to the Web of Data yet requires scalable and accurate approaches for the extraction of structured data in RDF (Resource Description Framework) from these websites. One of the key steps towards extracting RDF from text is the disambiguation of named entities. While several approaches aim to tackle this problem, they still achieve poor accuracy. We address this drawback by presenting AGDISTIS, a novel knowledge-base-agnostic approach for named entity disambiguation. Our approach combines the Hypertext-Induced Topic Search (HITS) algorithm with label expansion strategies and string similarity measures. Based on this combination, AGDISTIS can efficiently detect the correct URIs for a given set of named entities within an input text. We evaluate our approach on eight different datasets against state-of-the-art named entity disambiguation frameworks. Our results indicate that we outperform the state-of-the-art approach by up to 29 F-measure. Today people typically read and annotate printed documents even if they are obtained from electronic sources like digital libraries If there is a reason for them to share these personal annotations online, they must re-enter them. Given the advent of better computer support for reading and annotation, including tablet interfaces, will people ever share their personal digital ink annotations as is, or will they make substantial changes to them? What can we do to anticipate and support the transition from personal to public annotations? To investigate these questions, we performed a study to characterize and compare students' personal annotations as they read assigned papers with those they shared with each other using an online system. By analyzing over 1, 700 annotations, we confirmed three hypotheses: (1) only a small fraction of annotations made while reading are directly related to those shared in discussion; (2) some types of annotations - those that consist of anchors in the text coupled with margin notes - are more apt to be the basis of public commentary than other types of annotations; and (3) personal annotations undergo dramatic changes when they are shared in discussion, both in content and in how they are anchored to the source document. We then use these findings to explore ways to support the transition from personal to public annotations.