Given a relation instance \(s_i=(\mathbf {x}_i,h_i, t_i)\) , which consists of a sentence \(\mathbf {x}_i=\lbrace x_1,x_2,...,x_n\rbrace \)  and two entity spans \(h_i=(s_h,e_h)\) , \(t_i=(s_t,e_t)\)  marking the position of the entity pair, the entity pair encoder \(\mathbf {f}\)  aims to map relation instance \(s_i\)  to a fixed-length embedding \(\mathbf {h}_i=\mathbf {f}(s_i) \in \mathbb {R}^d\)  that encode contextual information in \(s_i\) . We adopt BERT [1]} as the implemention of our encoder \(\mathbf {f}\)  due to its strong performance on extracting contextual information. Formally:
\(\mathbf {h}_1^r,...,\mathbf {h}_n^r={\rm BERT}^r(x_1,...,x_n)\\\mathbf {h}_{ent}={\rm MAXPOOL}(\mathbf {h}_s^r,...,\mathbf {h}_e^r)\\\mathbf {h}_i=\mathbf {h}_{head}\oplus \mathbf {h}_{tail},\) 
