Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We briefly review Hochreiter's (1991) analysis of this problem, then address it by introducing a novel, efficient, gradient based method called long short-term memory (LSTM). Truncating the gradient where this does not do harm, LSTM can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units. Multiplicative gate units learn to open and close access to the constant error flow. LSTM is local in space and time; its computational complexity per time step and weight is O. 1. Our experiments with artificial data involve local, distributed, real-valued, and noisy pattern representations. In comparisons with real-time recurrent learning, back propagation through time, recurrent cascade correlation, Elman nets, and neural sequence chunking, LSTM leads to many more successful runs, and learns much faster. LSTM also solves complex, artificial long-time-lag tasks that have never been solved by previous recurrent network algorithms. <s> BIB001 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> This paper presents a syntax-driven approach to question answering, specifically the answer-sentence selection problem for short-answer questions. Rather than using syntactic features to augment existing statistical classifiers (as in previous work), we build on the idea that questions and their (correct) answers relate to each other via loose but predictable syntactic transformations. We propose a probabilistic quasi-synchronous grammar, inspired by one proposed for machine translation (D. Smith and Eisner, 2006), and parameterized by mixtures of a robust nonlexical syntax/alignment model with a(n optional) lexical-semantics-driven log-linear model. Our model learns soft alignments as a hidden variable in discriminative training. Experimentalresultsusing theTRECdataset are shown to significantly outperform strong state-of-the-art baselines. <s> BIB002 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> We describe tree edit models for representing sequences of tree transformations involving complex reordering phenomena and demonstrate that they offer a simple, intuitive, and effective method for modeling pairs of semantically related sentences. To efficiently extract sequences of edits, we employ a tree kernel as a heuristic in a greedy search routine. We describe a logistic regression model that uses 33 syntactic features of edit sequences to classify the sentence pairs. The approach leads to competitive performance in recognizing textual entailment, paraphrase identification, and answer selection for question answering. <s> BIB003 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> A range of Natural Language Processing tasks involve making judgments about the semantic relatedness of a pair of sentences, such as Recognizing Textual Entailment (RTE) and answer selection for Question Answering (QA). A key challenge that these tasks face in common is the lack of explicit alignment annotation between a sentence pair. We capture the alignment by using a novel probabilistic model that models tree-edit operations on dependency parse trees. Unlike previous tree-edit models which require a separate alignment-finding phase and resort to ad-hoc distance metrics, our method treats alignments as structured latent variables, and offers a principled framework for incorporating complex linguistic features. We demonstrate the robustness of our model by conducting experiments for RTE and QA, and show that our model performs competitively on both tasks with the same set of general features. <s> BIB004 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> The recently introduced continuous Skip-gram model is an efficient method for learning high-quality distributed vector representations that capture a large number of precise syntactic and semantic word relationships. In this paper we present several extensions that improve both the quality of the vectors and the training speed. By subsampling of the frequent words we obtain significant speedup and also learn more regular word representations. We also describe a simple alternative to the hierarchical softmax called negative sampling. ::: ::: An inherent limitation of word representations is their indifference to word order and their inability to represent idiomatic phrases. For example, the meanings of "Canada" and "Air" cannot be easily combined to obtain "Air Canada". Motivated by this example, we present a simple method for finding phrases in text, and show that learning good vector representations for millions of phrases is possible. <s> BIB005 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> Our goal is to extract answers from preretrieved sentences for Question Answering (QA). We construct a linear-chain Conditional Random Field based on pairs of questions and their possible answer sentences, learning the association between questions and answer types. This casts answer extraction as an answer sequence tagging problem for the first time, where knowledge of shared structure between question and source sentence is incorporated through features based on Tree Edit Distance (TED). Our model is free of manually created question and answer templates, fast to run (processing 200 QA pairs per second excluding parsing time), and yields an F1 of 63.3% on a new public dataset based on prior TREC QA evaluations. The developed system is open-source, and includes an implementation of the TED model that is state of the art in the task of ranking QA pairs. <s> BIB006 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> Recent methods for learning vector space representations of words have succeeded in capturing fine-grained semantic and syntactic regularities using vector arithmetic, but the origin of these regularities has remained opaque. We analyze and make explicit the model properties needed for such regularities to emerge in word vectors. The result is a new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods. Our model efficiently leverages statistical information by training only on the nonzero elements in a word-word cooccurrence matrix, rather than on the entire sparse matrix or on individual context windows in a large corpus. The model produces a vector space with meaningful substructure, as evidenced by its performance of 75% on a recent word analogy task. It also outperforms related models on similarity tasks and named entity recognition. <s> BIB007 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> We describe the WIKIQA dataset, a new publicly available set of question and sentence pairs, collected and annotated for research on open-domain question answering. Most previous work on answer sentence selection focuses on a dataset created using the TREC-QA data, which includes editor-generated questions and candidate answer sentences selected by matching content words in the question. WIKIQA is constructed using a more natural process and is more than an order of magnitude larger than the previous dataset. In addition, the WIKIQA dataset also includes questions for which there are no correct sentences, enabling researchers to work on answer triggering, a critical component in any QA system. We compare several systems on the task of answer sentence selection on both datasets and also describe the performance of a system on the problem of answer triggering using the WIKIQA dataset. <s> BIB008 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> We apply a general deep learning framework to address the non-factoid question answering task. Our approach does not rely on any linguistic tools and can be applied to different languages or domains. Various architectures are presented and compared. We create and release a QA corpus and setup a new QA task in the insurance domain. Experimental results demonstrate superior performance compared to the baseline methods and various technologies give further improvements. For this highly challenging task, the top-1 accuracy can reach up to 65.3% on a test set, which indicates a great potential for practical use. <s> BIB009 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> In this paper, we propose a novel word-alignment-based method to solve the FAQ-based question answering task. First, we employ a neural network model to calculate question similarity, where the word alignment between two questions is used for extracting features. Second, we design a bootstrap-based feature extraction method to extract a small set of effective lexical features. Third, we propose a learning-to-rank algorithm to train parameters more suitable for the ranking tasks. Experimental results, conducted on three languages (English, Spanish and Japanese), demonstrate that the question similarity model is more effective than baseline systems, the sparse features bring 5% improvements on top-1 accuracy, and the learning-to-rank algorithm works significantly better than the traditional method. We further evaluate our method on the answer sentence selection task. Our method outperforms all the previous systems on the standard TREC data set. <s> BIB010 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> Matching natural language sentences is central for many applications such as information retrieval and question answering. Existing deep models rely on a single sentence representation or multiple granularity representations for matching. However, such methods cannot well capture the contextualized local information in the matching process. To tackle this problem, we present a new deep architecture to match two sentences with multiple positional sentence representations. Specifically, each positional sentence representation is a sentence representation at this position, generated by a bidirectional long short term memory (Bi-LSTM). The matching score is finally produced by aggregating interactions between these different positional sentence representations, through $k$-Max pooling and a multi-layer perceptron. Our model has several advantages: (1) By using Bi-LSTM, rich context of the whole sentence is leveraged to capture the contextualized local information in each positional sentence representation; (2) By matching with multiple positional sentence representations, it is flexible to aggregate different important contextualized local information in a sentence to support the matching; (3) Experiments on different tasks such as question answering and sentence completion demonstrate the superiority of our model. <s> BIB011 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> We introduce the MovieQA dataset which aims to evaluate automatic story comprehension from both video and text. The dataset consists of 14,944 questions about 408 movies with high semantic diversity. The questions range from simpler "Who" did "What" to "Whom", to "Why" and "How" certain events occurred. Each question comes with a set of five possible answers; a correct one and four deceiving answers provided by human annotators. Our dataset is unique in that it contains multiple sources of information -- video clips, plots, subtitles, scripts, and DVS. We analyze our data through various statistics and methods. We further extend existing QA techniques to show that question-answering with such open-ended semantics is hard. We make this data set public along with an evaluation benchmark to encourage inspiring work in this challenging domain. <s> BIB012 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> Recent works using artificial neural networks based on distributed word representation greatly boost performance on various natural language processing tasks, especially the answer selection problem. Nevertheless, most of the previous works used deep learning methods (like LSTM-RNN, CNN, etc.) only to capture semantic representation of each sentence separately, without considering the interdependence between each other. In this paper, we propose a novel end-to-end learning framework which constitutes deep convolutional neural network based on multi-modal similarity metric learning (M$^2$S-Net) on pairwise tokens. The proposed model demonstrates its performance by surpassing previous state-of-the-art systems on the answer selection benchmark, i.e., TREC-QA dataset, in both MAP and MRR metrics. <s> BIB013 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> We study answer selection for question answering, in which given a question and a set of candidate answer sentences, the goal is to identify the subset that contains the answer. Unlike previous work which treats this task as a straightforward pointwise classification problem, we model this problem as a ranking task and propose a pairwise ranking approach that can directly exploit existing pointwise neural network models as base components. We extend the Noise-Contrastive Estimation approach with a triplet ranking loss function to exploit interactions in triplet inputs over the question paired with positive and negative examples. Experiments on TrecQA and WikiQA datasets show that our approach achieves state-of-the-art effectiveness without the need for external knowledge sources or feature engineering. <s> BIB014 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> Many NLP tasks including machine comprehension, answer selection and text entailment require the comparison between sequences. Matching the important units between sequences is a key to solve these problems. In this paper, we present a general "compare-aggregate" framework that performs word-level matching followed by aggregation using Convolutional Neural Networks. We particularly focus on the different comparison functions we can use to match two vectors. We use four different datasets to evaluate the model. We find that some simple comparison functions based on element-wise operations can work better than standard neural network and neural tensor network. <s> BIB015 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> Answer selection for question answering is a challenging task, since it requires effective capture of the complex semantic relations between questions and answers. Previous remarkable approaches mainly adopt general Compare-Aggregate framework that performs word-level comparison and aggregation. In this paper, unlike previous Compare-Aggregate models which utilize the traditional attention mechanism to generate corresponding word-level vector before comparison, we propose a novel attention mechanism named Dynamic-Clip Attention which is directly integrated into the Compare-Aggregate framework. Dynamic-Clip Attention focuses on filtering out noise in attention matrix, in order to better mine the semantic relevance of word-level vectors. At the same time, different from previous Compare-Aggregate works which treat answer selection task as a pointwise classification problem, we propose a listwise ranking approach to model this task to learn the relative order of candidate answers. Experiments on TrecQA and WikiQA datasets show that our proposed model achieves the state-of-the-art performance. <s> BIB016 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> We describe a new deep learning architecture for learning to rank question answer pairs. Our approach extends the long short-term memory (LSTM) network with holographic composition to model the relationship between question and answer representations. As opposed to the neural tensor layer that has been adopted recently, the holographic composition provides the benefits of scalable and rich representational learning approach without incurring huge parameter costs. Overall, we present Holographic Dual LSTM (HD-LSTM), a unified architecture for both deep sentence modeling and semantic matching. Essentially, our model is trained end-to-end whereby the parameters of the LSTM are optimized in a way that best explains the correlation between question and answer representations. In addition, our proposed deep learning architecture requires no extensive feature engineering. Via extensive experiments, we show that HD-LSTM outperforms many other neural architectures on two popular benchmark QA datasets. Empirical studies confirm the effectiveness of holographic composition over the neural tensor layer. <s> BIB017 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> For natural language understanding (NLU) technology to be maximally useful, both practically and as a scientific object of study, it must be general: it must be able to process language in a way that is not exclusively tailored to any one specific task or dataset. In pursuit of this objective, we introduce the General Language Understanding Evaluation benchmark (GLUE), a tool for evaluating and analyzing the performance of models across a diverse range of existing NLU tasks. GLUE is model-agnostic, but it incentivizes sharing knowledge across tasks because certain tasks have very limited training data. We further provide a hand-crafted diagnostic test suite that enables detailed linguistic analysis of NLU models. We evaluate baselines based on current methods for multi-task and transfer learning and find that they do not immediately give substantial improvements over the aggregate performance of training a separate model per task, indicating room for improvement in developing general and robust NLU systems. <s> BIB018 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> We introduce a new type of deep contextualized word representation that models both (1) complex characteristics of word use (e.g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i.e., to model polysemy). Our word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pre-trained on a large text corpus. We show that these representations can be easily added to existing models and significantly improve the state of the art across six challenging NLP problems, including question answering, textual entailment and sentiment analysis. We also present an analysis showing that exposing the deep internals of the pre-trained network is crucial, allowing downstream models to mix different types of semi-supervision signals. <s> BIB019 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> In this paper, we propose a novel method for a sentence-level answer-selection task that is a fundamental problem in natural language processing. First, we explore the effect of additional information by adopting a pretrained language model to compute the vector representation of the input text and by applying transfer learning from a large-scale corpus. Second, we enhance the compare-aggregate model by proposing a novel latent clustering method to compute additional information within the target corpus and by changing the objective function from listwise to pointwise. To evaluate the performance of the proposed approaches, experiments are performed with the WikiQA and TREC-QA datasets. The empirical results demonstrate the superiority of our proposed approach, which achieve state-of-the-art performance for both datasets. <s> BIB020 </s> Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Survey <s> Word representation layer: Each word is represented with a d-dimensional <s> Recently, pre-trained models have been the dominant paradigm in natural language processing. They achieved remarkable state-of-the-art performance across a wide range of related tasks, such as textual entailment, natural language inference, question answering, etc. BERT, proposed by Devlin this http URL., has achieved a better marked result in GLUE leaderboard with a deep transformer architecture. Despite its soaring popularity, however, BERT has not yet been applied to answer selection. This task is different from others with a few nuances: first, modeling the relevance and correctness of candidates matters compared to semantic relatedness and syntactic structure; second, the length of an answer may be different from other candidates and questions. In this paper. we are the first to explore the performance of fine-tuning BERT for answer selection. We achieved STOA results across five popular datasets, demonstrating the success of pre-trained models in this task. <s> BIB021
vector constructed by a word embedding and a character-composed embedding. Word embeddings are obtained from pre-trained GloVe BIB007 or word2vec BIB005 embeddings. Charactercomposed embeddings are generated by feeding characters of words into an LSTM BIB001 . 2. Context representation layer: A BiLSTM is used in order to combine contextual information of a sentence with its representation. 3. Matching layer: In this layer, each contextual embedding of one sentence is compared with all the contextual representations of the other sentence using a multi-perspective matching operation. Also question and answer sentences are matched in two directions. Multi-perspective cosine matching function is defined as: where v 1 and v 2 are d-dimensional vectors, W ∈ R l×d is trainable parameter and each perspective is controlled by one row of W , and M is a l-dimensional vector. Each M k is calculated as follows: let w k be k-th row of W and • be element-wise multiplication. Four different matching strategies are proposed based on different f m functions. These matching strategies are shown in Figure 28 and described just for one direction in the following. -Full-Matching: Each forward time step representation of the answer sentence − → h p i is compared with every forward time step representations of the question sentence − → h q N . This strategy is shown in Figure 28 (a) . -Maxpooling-Matching: Maximum similarity for each forward time step representation of the answer sentence with all the time steps of the forward representation of the question sentence is returned. This strategy is shown in Figure 28 (b) . -Attentive-Matching: Cosine similarity between each forward time step representation of the answer sentence − → h p i and each forward time step representation of the question sentence − → h q i is considered as attention weight. This strategy is shown in Figure 28 (c) . Then the new representation of the question sentence (Q) called h mean i is generated by calculating the weighted average of its forward time steps representations by using attention weights. And finally, the matching vector for each time step representation of answer sentence is calculated with its corresponding attentive vector h mean i . -Max-Attentive-Matching: This strategy is different from attentive-matching strategy just in generating attentive vector (h mean i ). Attentive vector here is the contextual embedding with the highest similarity. This strategy is shown in Figure 28 (d) . Finally for each direction all of these strategies are applied for each time-step and eight generated vectors are concatenated and considered as the matching in that direction. obtained from matching layer, are fed into a BiLSTM. Then a fixed-length matching vector is obtained by concatenating the last four output vectors of two BiLSTMs. 5. Prediction layer: In this layer, P (y|P, Q) is predicted using a two layer feed-forward neural network followed by a softMax layer. The fixed length matching-vector is fed to this layer. Wang and Jiang (2017) proposed a compare-aggregate model for matching two sentences. This model for each pair of Q and A predicts a label y which shows whether the candidate answer A is a correct answer for question Q or not. According to the architecture of this model, which is shown in the Figure  29 , this model consists four following major layer: 1. Preprocessing layer: This layer constructs an embedding for each word which represents the word and its contextual information. Q and A are inputs of this layer. A version of LSTM/GRU, which uses only the input gates, is applied for generating Q ∈ R l×Q and A ∈ R l×A matrices. where W i , W u ∈ R l×d and b i , b u ∈ R l are parameters, and (. ⊗ e X ) generates a matrix by repeating the vector on the left for X times. 2. Attention layer: This layer is applied on output of the previous layer. Attention-weighted vector H ∈ R l×A is obtained by the following equations. The j th column of H indicates the part of Q that best matches the j th word in A. where W g ∈ R l×l and b g ∈ R are parameters, and G ∈ R Q×A is the attention weight matrix. 3. Comparison layer: Embedding of each word a j in answer is matched with the corresponding attention weight h j and the comparison result is indicated with vector t j . In this work, six different comparison functions are introduced. -Neural Net (NN): -Neural Tensor Net (NTN): -Euclidean distance or cosine similarity (EucCos): -Subtraction (Sub): -Multiplication (Mult): -Submult + NN: Among these comparison functions NN and NTN do not capture the similarity well. EucCos may ignore some important information. Sub and Mult are similar to the Euclidean distance and Cosine similarity, and the last model is the combination of the Sub, Mult, and NN. 4. Aggregation: A one-layer CNN is used for combining t j vectors. Output of the aggregation is r ∈ R nl which is used in the final classifier. Tay et al. (2018) proposed Multi-Cast Attention Network (MCAN) for retrieval-based QA. Inputs of MCAN are two sentences: question q and document d sentences. As is shown in Figure 30 , MCAN has five major layers. These layers are described in the following. 1. Input Encoder: Input sentences are fed to this network as one-hot encoded vectors and word embeddings are generated by passing through an embedding layer. Highway encoder like RNNs by using gating mechanism controls the flow of the information. A highway encoder layer is used for detecting important and not important words in given sentence. A single highway network is formulated as: where H(.) and T (.) are one-layer affine transforms with ReLU and sigmoid activation functions, and W H , W T ∈ R r×d . 2. Co-Attention: In this layer a similarity matrix which denotes the similarity between each pair of words across both sentences is learned by the following formulations: where F could be a multi-layered perceptron .  where Soft is softMax function, and d and q are co-attentive representations of the document and question. Performance of these poolings varies on different datasets, but in general max-pooling pays attention to words based on their maximum influence and mean-pooling pays attention to words based on their total influence on the words of the other sentence. 2.2. Alignment pooling: Word pairs from two sentences are realigned in this pooling strategy. Co-attentive representations are learned as below: let d i be the sub-phrase of q which is aligned to d i . 2.3. Intra-attention: Intra-attention attempts to represent long-term dependencies in one sentence. Representation of each sentence is learned regardless of the other sentence. So, it is applied on both the document and the question separately. Co-attentive representations are learned as below:  where x denotes the co-attention representation of x, and F C is a compression function. In the above formulations x and x are compared by three different operators for modeling the difference between x and x from different perspectives. Difference between x and x is an n-dimensional vector which is compressed by a compression function to a scalar. Three different compression functions: sum, fully-connected layer, and Factorization Machines (FM) are used. As is shown in Figure 30 , given a document question pair, co-attention with three different poolings (1) mean-pooling, (2) Max-pooling, (3) alignment-pooling are applied on pair of question and document and (4) Intra-attention is applied on document and question separately. 12 scalars are generated for each word and concatenated with word embedding. Then each word w i is represented as w i = [w i ; z i ] where z ∈ R 12 is output of multi-cast layer. 4. LSTM encoder: Casted representation of words of a sentence which are generated by multi-cast attention are fed to an LSTM encoder, and a meanMax pooling is applied on hidden states of the LSTM. Casted representations of a words helps the LSTM network with its knowledge about each sentence and between question and document, in extracting long-term dependencies. 5. Prediction layer and optimization: Given representation of the document and the question, prediction is computed by using two-layer highway network and a softMax layer as follows: where H 1 , and H 2 are highway network layers with ReLU activation and BIB020 proposed CompClip for predicting matching score (y) of given pair of question (Q = q 1 , ..., q n ) and answer (A = a 1 , ..., a n ). For improving the performance of CompClip, they have applied transfer learning technique by training it on question-answering NLI (QNLI) corpus BIB018 . They have also used pointwise learning to rank approach for training this model. The most prominent feature of their work is using ELMo language model for achieving more meaningful contextual information of question and answer sentences. Architecture of CompClip consists of six layers, as is illustrated in Figure 31 . These layers are described below. 1. Language model: Instead of using a word embedding layer, ELMo language model BIB019 is used for extracting contextual information of given sentence in a more efficient way. After applying ELMo language model, new representation of question and answer are denoted as L Q and L A , respectively. 2. Context representation: This part of model learns the weight W for extracting contextual information of given sentence and generating its contextual representation by following equations: after applying Elmo language model, Q and A are replaced by L Q and L A , respectively. 3. Attention: Attentional representation of question H Q and answer H A sentences are generated utilizing dynamic-clip attention BIB016 as follows:  5. Aggregation layer: For aggregating outputs of comparison layer, a CNN with n-types of filters is employed. Aim of this layer is computing the matching score (score) between question and answer as follows:  Available Results from the Literature As mentioned in Section 2, WikiQA BIB008 , TREC-QA BIB002 , MovieQA BIB012 , InsuranceQA BIB009 , and Yahoo! (Wan et al., 2016b,a) are the main datasets that have been used for evaluating text-based QA systems. In this section, we report the results of different models on the mentioned datasets to have a comparison on the quality of the state-of-the-art models. It has to be mentioned that considering a large number of models reviewed in this paper, it is not possible to reimplement all of them to have a comprehensive comparison and we only report the results based upon their availability. Table 5 reports the results of different representation-based (R), interactionbased (I), and hybrid (H) methods on the WikiQA dataset and compares it with other baseline models. As can be seen BERTSel BIB021 achieved the best results over all methods on the WikiQA dataset in MRR and MAP metrics. Although this interaction-based technique is the best sate-of-the-art model in the field, comparing the rest of models, we can see that the general performance of hybrid models are better than interaction-based models and the next best results are all from the hybrid models. Table 6 reports the results of different models which are described in Section 6 as well as their baseline methods on TREC-QA dataset. According to this table, the proposed model by BIB020 which uses ELMo language model, latent clustering, as well as transfer learning achieved best results over all models on the TREC-QA dataset in MRR. The BERTSel BIB021 model achieved the best result in MAP. BIB002 0.6029 0.6852 BIB003 0.6091 0.6917 BIB004 0.5951 0.6951 BIB006 0.6307 0.7477 BIB009 Architecture-II 0.711 0.800 BIB010 0.746 0.820 dos 0.7530 0.8511 BIB013 0.779 0.848 BIB014 0 Performance of different models from Section 6 and their baselines on Yahoo! dataset is reported in Table 7 . One representation-based model BIB017 , two interaction-based models BIB018 BIB021 , and one hybrid model BIB011 are evaluated on the Yahoo! dataset. As can be seen, Bi-Match-SRNN, proposed by BIB018 and BERT-Sel, proposed by BIB021 outperforms other models in P@1 and MRR metrics, respectively. Except BERTSel, which has superior performance on all datasets, this is the only dataset in which an interaction-based model, namely Bi-Match-SRNN, performs better than hybrid model. Table 8 reports the results of different models on Insurance-QA dataset. Among three categories of deep models (representation-based, interactionbased, and hybrid models) hybrid models achieves the best accuracy on Insurance-QA dataset too. Proposed model by BIB015 with SUBMULT+NN comparison function on TEST1, and with Mult comparison function on TEST2 achieves the best accuracy.