[1]} used a bidirectional gated recurrent unit (Bi-GRU) [2]} to capture the long-term dependency among the words in the sentence. The tokens vectors \(\mathbf {x}_t\)  are passed to a Bi-GRU layer. The hidden vectors of the Bi-GRU layer are passed to a bi-linear operator which is a combination of two feed-forward layers with softmax to compute a scalar attention score for each word. The hidden vectors of the Bi-GRU layer are multiplied by their corresponding attention scores for scaling up the hidden vectors. A piecewise convolution neural network [3]} is applied to the scaled hidden vectors to obtain the feature vector. This feature vector is passed to a feed-forward layer with softmax to determine the relation.
