State-of-the-art networks that model relations between two pieces of text often use complex architectures and attention . In this paper , instead of focusing on architecture engineering , we take advantage of small amounts of labelled data that model semantic phenomena in text to encode matching features directly in the word representations . This greatly boosts the accuracy of our reference network , while keeping the model simple and fast to train . Our approach also beats a tree kernel model that uses similar input encodings , and neural models which use advanced attention and compare-aggregate mechanisms . Modeling a match between pieces of text is at the core of many NLP tasks . Recently , manual feature engineering methods have been shadowed by neural network approaches . These networks model the interaction of two pieces of text , or word-toword interactions across sentences , using sophisticated attention mechanisms ( Wang et al . , 2016a ; Santos et al . , 2016 ) and compare-aggregate frameworks ( He and Lin , 2016 ; Wang et al . , 2017 ) . Architectural complexity is tied to longer training times 1 . Meaningful features may take long time to emerge by only leveraging word representations and the training data of the task at hand . This is especially problematic with little data , as it often happens in question answering ( QA ) tasks , e.g. , answer sentence selection ( Wang et al . , 2007 ; Yang et al . , 2015 ) . Thus , effective word representations are crucial in neural network models to get state-of-the-art performance . In this work , we try to answer the following research questions : ( i ) in addition to lexical links , can we incorporate higher-level semantic links between the words in a question and a candidate answer passage , and ( ii ) can we show that such information has an impact on the quality of our model , and also allows us to keep the architecture simple ? We show that modeling semantic relations improves the performance of a neural network for answer sentence selection with ( i ) a little number of semantic annotations , and ( ii ) a little increase in training time w.r.t . more complex architecture . In this paper , we presented a neural network that models semantic links between questions and answers , in addition to lexical links . The annotations for establishing such links are produced by a set of fast neural components for question analysis , trained on publicly available datasets . The evaluation on two QA datasets shows that our approach can achieve state-of-the-art performance using a simple CNN , leading to a low complexity and training time . Our approach is an interesting first step towards a future architecture , in which we will jointly optimize the semantic annotators and the answer sentence selection model , in an end-to-end fashion .
State-of-the-art models that model relations between two texts use complex architectures and attention which requires a long time and large data at training.
They propose a method that directly models higher-level semantic links between two texts that are annotated by a fast model.
The proposed model outperforms a tree kernel model and complex neural models while keeping the model simple and the training fast.