Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Question Answer Similarity from Information Retrieval Perspective <s> is one of the most recognizablecharacters in 20th century cinema. HAL is an artiﬁcial agent capable of such advancedlanguage behavior as speaking and understanding English, and at a crucial moment inthe plot, even reading lips. It is now clear that HAL’s creator, Arthur C. Clarke, wasa little optimistic in predicting when an artiﬁcial agent such as HAL would be avail-able. But just how far off was he? What would it take to create at least the language-relatedpartsofHAL?WecallprogramslikeHALthatconversewithhumansinnatural <s> BIB001 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Question Answer Similarity from Information Retrieval Perspective <s> In this paper we propose a term clustering approach to improve the performance of sentence retrieval in Question Answering (QA) systems. As the search in question answering is conducted over smaller segments of data than in a document retrieval task, the problems of data sparsity and exact matching become more critical. In this paper we propose Language Modeling (LM) techniques to overcome such problems and improve the sentence retrieval performance. Our proposed methods include building class-based models by term clustering, and then employing higher order n-grams with the new class-based model. We report our experiments on the TREC 2007 questions from QA track. The results show that the methods investigated here enhanced the mean average precision of sentence retrieval from 23.62% to 29.91%. <s> BIB002 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Question Answer Similarity from Information Retrieval Perspective <s> This paper proposes a general learning framework for a class of problems that require learning over latent intermediate representations. Many natural language processing (NLP) decision problems are defined over an expressive intermediate representation that is not explicit in the input, leaving the algorithm with both the task of recovering a good intermediate representation and learning to classify correctly. Most current systems separate the learning problem into two stages by solving the first step of recovering the intermediate representation heuristically and using it to learn the final classifier. This paper develops a novel joint learning algorithm for both tasks, that uses the final prediction to guide the selection of the best intermediate representation. We evaluate our algorithm on three different NLP tasks -- transliteration, paraphrase identification and textual entailment -- and show that our joint method significantly improves performance. <s> BIB003 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Question Answer Similarity from Information Retrieval Perspective <s> In this paper, we study the answer sentence selection problem for question answering. Unlike previous work, which primarily leverages syntactic analysis through dependency tree matching, we focus on improving the performance using models of lexical semantic resources. Experiments show that our systems can be consistently and significantly improved with rich lexical semantic information, regardless of the choice of learning algorithms. When evaluated on a benchmark dataset, the MAP and MRR scores are increased by 8 to 10 points, compared to one of our baseline systems using only surface-form matching. Moreover, our best system also outperforms pervious work that makes use of the dependency tree structure by a wide margin. <s> BIB004 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Question Answer Similarity from Information Retrieval Perspective <s> Many machine learning algorithms require the input to be represented as a fixed-length feature vector. When it comes to texts, one of the most common fixed-length features is bag-of-words. Despite their popularity, bag-of-words features have two major weaknesses: they lose the ordering of the words and they also ignore semantics of the words. For example, "powerful," "strong" and "Paris" are equally distant. In this paper, we propose Paragraph Vector, an unsupervised algorithm that learns fixed-length feature representations from variable-length pieces of texts, such as sentences, paragraphs, and documents. Our algorithm represents each document by a dense vector which is trained to predict words in the document. Its construction gives our algorithm the potential to overcome the weaknesses of bag-of-words models. Empirical results show that Paragraph Vectors outperforms bag-of-words models as well as other techniques for text representations. Finally, we achieve new state-of-the-art results on several text classification and sentiment analysis tasks. <s> BIB005 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Question Answer Similarity from Information Retrieval Perspective <s> Recent methods for learning vector space representations of words have succeeded in capturing fine-grained semantic and syntactic regularities using vector arithmetic, but the origin of these regularities has remained opaque. We analyze and make explicit the model properties needed for such regularities to emerge in word vectors. The result is a new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods. Our model efficiently leverages statistical information by training only on the nonzero elements in a word-word cooccurrence matrix, rather than on the entire sparse matrix or on individual context windows in a large corpus. The model produces a vector space with meaningful substructure, as evidenced by its performance of 75% on a recent word analogy task. It also outperforms related models on similarity tasks and named entity recognition. <s> BIB006 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Question Answer Similarity from Information Retrieval Perspective <s> We describe the WIKIQA dataset, a new publicly available set of question and sentence pairs, collected and annotated for research on open-domain question answering. Most previous work on answer sentence selection focuses on a dataset created using the TREC-QA data, which includes editor-generated questions and candidate answer sentences selected by matching content words in the question. WIKIQA is constructed using a more natural process and is more than an order of magnitude larger than the previous dataset. In addition, the WIKIQA dataset also includes questions for which there are no correct sentences, enabling researchers to work on answer triggering, a critical component in any QA system. We compare several systems on the task of answer sentence selection on both datasets and also describe the performance of a system on the problem of answer triggering using the WIKIQA dataset. <s> BIB007 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Question Answer Similarity from Information Retrieval Perspective <s> We propose a new probabilistic approach to information retrieval based upon the ideas and methods of statistical machine translation. The central ingredient in this approach is a statistical model of how a user might distill or "translate" a given document into a query. To assess the relevance of a document to a user's query, we estimate the probability that the query would have been generated as a translation of the document, and factor in the user's general preferences in the form of a prior distribution over documents. We propose a simple, well motivated model of the document-to-query translation process, and describe an algorithm for learning the parameters of this model in an unsupervised manner from a collection of documents. As we show, one can view this approach as a generalization and justification of the "language modeling" strategy recently proposed by Ponte and Croft. In a series of experiments on TREC data, a simple translation-based retrieval system performs well in comparison to conventional retrieval techniques. This prototype system only begins to tap the full potential of translation-based retrieval. <s> BIB008 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Question Answer Similarity from Information Retrieval Perspective <s> While increasingly complex approaches to question answering (QA) have been proposed, the true gain of these systems, particularly with respect to their expensive training requirements, can be inflated when they are not compared to adequate baselines. Here we propose an unsupervised, simple, and fast alignment and information retrieval baseline that incorporates two novel contributions: a \textit{one-to-many alignment} between query and document terms and \textit{negative alignment} as a proxy for discriminative information. Our approach not only outperforms all conventional baselines as well as many supervised recurrent neural networks, but also approaches the state of the art for supervised systems on three QA datasets. With only three hyperparameters, we achieve 47\% P@1 on an 8th grade Science QA dataset, 32.9\% P@1 on a Yahoo! answers QA dataset and 64\% MAP on WikiQA. We also achieve 26.56\% and 58.36\% on ARC challenge and easy dataset respectively. In addition to including the additional ARC results in this version of the paper, for the ARC easy set only we also experimented with one additional parameter -- number of justifications retrieved. <s> BIB009
Although lexical-based information retrieval models have been widely used in ad-hoc retrieval, they have less applied to QA tasks because the length of BIB001 answer sentences is shorter than normal web documents and the vocabulary gap between question and answer sentence in QA is more pronoun than adhoc retrieval. It motivated researches to use advanced information retrieval approaches in QA. Some of these approaches are described in this section. BIB007 presented the WikiQA dataset for open domain QA. They have evaluated WikiQA and QASent datasets with information retrievalbased models like Word Count (Word Cnt), Weighted Word Count (Wgt Word Cnt), Learning Constrained Latent Representation (LCLR) BIB004 , and Paragraph Vector (PV) BIB005 . Word Cnt model works by counting the non-stop words in question which also have occurred in the answer sentence. Wgt Word Cnt is the same as Word Cnt, but it also reweights the counts by Inverse documenr Frequency (IDF) weight of the question words. The Idea behind the model proposed by BIB004 is adopting a probabilistic classifier for predicting whether a pair of question and answer are related or not, using semantic model of the question and answer. They used synonym/antonym, hypernym/hyponym and semantic word similarity of each pair of words from question and answer sentences for creating the semantic model. They adopted Learning Constrained Latent Representation (LCLR) BIB003 for classifying pairs of question and answer. Details of this classifier is shown in the following equations. Murdock and Croft (2004) suggested a translation model for QA. In their model, probability of the question (Q) given the answer (A), denoted as P (Q|A), is computed by the following equations: where λ is the smoothing parameter, D A is the document containing answer A, and C is the collection. The idea is based on the model proposed by BIB008 while different similarity model is used for calculating P (q i |a j ). BIB002 proposed class-based language models for sentence retrieval in QA. Their class-based language model aims to mitigate the word mismatch problem by finding the relation between words. The Brown word clustering algorithm is adopted for clustering the words in this model and the probability of generating question Q, having the answer sentence S is calculated by the following equations: is the probability of term q i having its cluster (C qi ) and answer sentence model (S), P (C qi |S) is the probability of cluster C qi given the sentence model S, f s (C qi ) is the number of occurrences of all the words in the cluster of term q i in sentence S, and w represents the vocabulary words. Klakow (2011, 2015) proposed a trained trigger language model. In their model, the word mismatch problem is mitigated by using the contextual information between words. The Idea behind this model is to find the pair of trigger and target words, whereas appearance of a target word in answer sentence and trigger word in question sentence means as a relation between the related words. They have trained a model for extracting these trigger-target pairs from a corpus, and this model is used for calculating the probability of question word q i , having the answer S, denoted as P (q i |S), as follows: where s j and q i denote the j th term in answer sentence, and i th term in the question sentence, respectively. f c (q i , s j ) is the number of times term q i triggers term s j in the model created upon corpus C. P (q i |S) is the probability of q i having the sentence S, and N is sentence length. Having the above probabilities, the probability of question (Q), having the answer( S), is calculated as follows: where M is question length. BIB009 proposed a model which for each question-answer pair calculates a matching score in three steps. In the first step, IDF weight of each word is calculated by the following equation: where N is the count of questions, and docf req(q i ) is the number of questions which word q i has occurred in. In the second step, one-to-many alignments are performed between terms in question and answer. Cosine similarity between Glove word embedding BIB006 of each question word q i and each answer word a i is considered as their similarity. Then the top K + most similar words {a .., a − q i ,K + } of answer are found. Finally in the third step the similarity score S(Q, A) between each question and answer sentence is calculated by the following equations: where align(q i , A) is the alignment score between q i and answer A, λ is the negative information's weight, pos(q i , A) and neg(q i , A) represent the one-tomany alignment score for the K + most and K − least similar words. They have also proposed two other baselines single-alignment (one-to-one), and one-toall. In one-to-one approach, just the single alignment score K + = 1, the most similar word, is used. In the one-to-all approach, similarity between the question term q i and all the answer terms is considered with the same weight in calculating align(q i , A) which changes the align(q i , A) to the following equation: where M is the count of the words in the answer sentence.