(4) BiGRU Word Attention (BGWA) [1]}: Words are represented using word embeddings and two positional embeddings. They are passed to a bidirectional gated recurrent unit (BiGRU) [2]} layer. Hidden vectors of the BiGRU layer are passed to a bilinear operator (a combination of two feed-forward layers) to compute a scalar attention score for each word. Hidden vectors of the BiGRU layer are multiplied by their corresponding attention scores. A piece-wise CNN is applied on the weighted hidden vectors to obtain the feature vector. This feature vector is passed to a feed-forward layer with softmax to determine the relation.
