Investigative journalism in recent years is confronted with two major challenges: 1) vast amounts of unstructured data originating from large text collections such as leaks or answers to Freedom of Information requests, and 2) multi-lingual data due to intensified global cooperation and communication in politics, business and civil society. Faced with these challenges, journalists are increasingly cooperating in international networks. To support such collaborations, we present the new version of new s leak 2.0, our open-source software for content-based searching of leaks. It includes three novel main features: 1) automatic language detection and language-dependent information extraction for 40 languages, 2) entity and keyword visualization for efficient exploration, and 3) decentral deployment for analysis of confidential data from various formats. We illustrate the new analysis capabilities with an exemplary case study.
The @cite_6 @cite_11 @cite_0 system is a third tool that supports investigative analysis of textual documents. Jigsaw focuses on the extraction of entities (using multiple NER tools for English) and their correlation with metadata. It provides visualization of entities as lists and document contents as a (tree-structured) word graph. instead visualizes coherence of such information as network graphs.
@cite_0 @cite_6 @cite_11
2165662062 1984416789 2023736097 2169612737
Investigative analysts who work with collections of text documents connect embedded threads of evidence in order to formulate hypotheses about plans and activities of potential interest. As the number of documents and the corresponding number of concepts and entities within the documents grow larger, sense-making processes become more and more difficult for the analysts. We have developed a visual analytic system called Jigsaw that represents documents and their entities visually in order to help analysts examine reports more efficiently and develop theories about potential actions more quickly. Jigsaw provides multiple coordinated views of document entities with a special emphasis on visually illustrating connections between entities across the different documents. Analyzing and understanding collections of textual documents is an important task for professional analysts and a common everyday scenario for nonprofessionals. We have developed the Jigsaw visual analytics system to support these types of sensemaking activities. Jigsaw’s development benefited significantly from the existence of the VAST Contest Challenge that provided (1) diverse document collections to use as examples, (2) controlled exercises with a set of analytic tasks and solutions for judging results, and (3) visibility and publicity to help communicate our ideas to others. This article describes our participation in a series of VAST Contest Challenge efforts and how this participation helped influence Jigsaw’s design and development. We describe how the system’s capabilities have evolved over time, and we identify the particular lessons that we learned by participating in the challenges. EDGAR (Extraction of Drugs, Genes and Relations) is a natural language processing system that extracts information about drugs and genes relevant to cancer from the biomedical literature. This automatically extracted information has remarkable potential to facilitate computational analysis in the molecular biology of cancer, and the technology is straightforwardly generalizable to many areas of biomedicine. This paper reports on the mechanisms for automatically generating such assertions and on a simple application, conceptual clustering of documents. The system uses a stochastic part of speech tagger, generates an underspecified syntactic parse and then uses semantic and pragmatic information to construct its assertions. The system builds on two important existing resources: the MEDLINE database of biomedical citations and abstracts and the Unified Medical Language System, which provides syntactic and semantic information about the terms found in biomedical abstracts. Effective extraction of query relevant information present within documents on the web is a nontrivial task. In this paper we present our system called QueSTS, which does the above task by filtering and aggregating important query relevant sentences distributed across a set of documents. Our approach captures the contextual relationships among sentences of all input documents and represents them as an “integrated graph”. These relationships are exploited and several subgraphs of integrated graph which consist of sentences that are highly relevant to the query and that are highly related to each other are constructed. These subgraphs are ranked by our scoring model. The highest ranked subgraph which is rich in query relevant information and also has sentences that are highly coherent is returned as a query specific summary.