Recently, attention networks have proven very useful for different NLP tasks. [1]} and [2]} used word-level attention model for single-instance sentence-level relation extraction. [1]} proposed a combination of a convolutional neural network model and an attention network. First, a convolution operation with max-pooling is used to extract the global features of the sentence. Next, attention is applied to the words of the sentence based on the two entities separately. The word embedding of the last token of an entity is concatenated with the embedding of every word. This concatenated representation is passed to a feed-forward layer with tanh activation and then another feed-forward layer with softmax to get a scalar attention score for every word of that entity. The word embeddings are averaged based on the attention scores to get the attentive feature vectors. The global feature vector and two attentive feature vectors for the two entities are concatenated and passed to a feed-forward layer with softmax to determine the relation.
