Question Answering (QA), as a research field, has primarily focused on either knowledge bases (KBs) or free text as a source of knowledge. These two sources have historically shaped the kinds of questions that are asked over these sources, and the methods developed to answer them. In this work, we look towards a practical use-case of QA over user-instructed knowledge that uniquely combines elements of both structured QA over knowledge bases, and unstructured QA over narrative, introducing the task of multi-relational QA over personal narrative. As a first step towards this goal, we make three key contributions: (i) we generate and release TextWorldsQA, a set of five diverse datasets, where each dataset contains dynamic narrative that describes entities and relations in a simulated world, paired with variably compositional questions over that knowledge, (ii) we perform a thorough evaluation and analysis of several state-of-the-art QA models and their variants at this task, and (iii) we release a lightweight Python-based framework we call TextWorlds for easily generating arbitrary additional worlds and narrative, with the goal of allowing the community to create and share a growing collection of diverse worlds as a test-bed for this task.
proposes to use synthetic QA tasks (the bAbI dataset) to better understand the limitations of QA systems. bAbI builds on a simulated physical world similar to interactive fiction @cite_30 with simple objects and relations and includes 20 different reasoning tasks. Various types of end-to-end neural networks @cite_29 @cite_39 @cite_31 have demonstrated promising accuracies on this dataset. However, the performance can hardly translate to real-world QA datasets, as bAbI uses a small vocabulary (150 words) and short sentences with limited language variations (e.g., nesting sentences, coreference). A more sophisticated QA dataset with a supporting KB is WikiMovies @cite_44 , which contains 100k questions about movies, each of them is answerable by using either a KB or a Wikipedia article. However, WikiMovies is highly domain-specific, and similar to bAbI , the questions are designed to be in simple forms with little compositionality and hence limit the difficulty level of the tasks.
@cite_30 @cite_29 @cite_39 @cite_44 @cite_31
1511456413 2175874768 2409591106 2133585753
From the Publisher: Interactive fiction--the best-known form of which is the text game or text adventure--has not received as much critical attention as have such other forms of electronic literature as hypertext fiction and the conversational programs known as chatterbots. Twisty Little Passages (the title refers to a maze in Adventure, the first interactive fiction) is the first book-length consideration of this form, examining it from gaming and literary perspectives. Nick Montfort, an interactive fiction author himself, offers both aficionados and first-time users a way to approach interactive fiction that will lead to a more pleasurable and meaningful experience of it. Twisty Little Passages looks at interactive fiction beginning with its most important literary ancestor, the riddle. Montfort then discusses Adventure and its precursors (including the I Ching and Dungeons and Dragons), and follows this with an examination of mainframe text games developed in response, focusing on the most influential work of that era, Zork. He then considers the introduction of commercial interactive fiction for home computers, particularly that produced by Infocom. Commercial works inspired an independent reaction, and Montfort describes the emergence of independent creators and the development of an online interactive fiction community in the 1990s. Finally, he considers the influence of interactive fiction on other literary and gaming forms. With Twisty Little Passages Nick Montfort places interactive fiction in its computational and literary contexts, opening up this still-developing form to new consideration. Question answering tasks have shown remarkable progress with distributed vector representation. In this paper, we investigate the recently proposed Facebook bAbI tasks which consist of twenty different categories of questions that require complex reasoning. Because the previous work on bAbI are all end-to-end models, errors could come from either an imperfect understanding of semantics or in certain steps of the reasoning. For clearer analysis, we propose two vector space models inspired by Tensor Product Representation (TPR) to perform knowledge encoding and logical reasoning based on common-sense inference. They together achieve near-perfect accuracy on all categories including positional reasoning and path finding that have proved difficult for most of the previous approaches. We hypothesize that the difficulties in these categories are due to the multi-relations in contrast to uni-relational characteristic of other categories. Our exploration sheds light on designing more sophisticated dataset and moving one step toward integrating transparent and interpretable formalism of TPR into existing learning paradigms. Directly reading documents and being able to answer questions from them is an unsolved challenge. To avoid its inherent difficulty, question answering (QA) has been directed towards using Knowledge Bases (KBs) instead, which has proven effective. Unfortunately KBs often suffer from being too restrictive, as the schema cannot support certain types of answers, and too sparse, e.g. Wikipedia contains much more information than Freebase. In this work we introduce a new method, Key-Value Memory Networks, that makes reading documents more viable by utilizing different encodings in the addressing and output stages of the memory read operation. To compare using KBs, information extraction or Wikipedia documents directly in a single framework we construct an analysis tool, WikiMovies, a QA dataset that contains raw text alongside a preprocessed KB, in the domain of movies. Our method reduces the gap between all three settings. It also achieves state-of-the-art results on the existing WikiQA benchmark. We propose Neural Reasoner, a framework for neural network-based reasoning over natural language sentences. Given a question, Neural Reasoner can infer over multiple supporting facts and find an answer to the question in specific forms. Neural Reasoner has 1) a specific interaction-pooling mechanism, allowing it to examine multiple facts, and 2) a deep architecture, allowing it to model the complicated logical relations in reasoning tasks. Assuming no particular structure exists in the question and facts, Neural Reasoner is able to accommodate different types of reasoning and different forms of language expressions. Despite the model complexity, Neural Reasoner can still be trained effectively in an end-to-end manner. Our empirical studies show that Neural Reasoner can outperform existing neural reasoning systems with remarkable margins on two difficult artificial tasks (Positional Reasoning and Path Finding) proposed in [8]. For example, it improves the accuracy on Path Finding(10K) from 33.4 [6] to over 98 .