In this section, we present our span representations by using a bidirectional encoder representations from transformers (BERT) model [1]}.
Figure REF  shows the model structure used to generate span representations \(m_i\)  (\(0 \le i \le k\) ) for all valid mention entities, each mention entity may contain one or several tokens \(w_t\) . For each token \(w_t\)  in the sentence, its embedding \(o_t\)  is generated by the BERT embedding layer. The generated embeddings are then passed to an attention layer with their outputs as the head-finding attention vetctors \(w_i^{head}\)  for the mention spans:
<FIGURE>\(\begin{split}o_{t}&={v}_{o}^Tf_{bert}(w_t)\\{\alpha }_{i,t}&=\dfrac{e^{o_t}}{\sum _{k=i_{start}}^{i_{end}}e^{o_k}}\\w_i^{head}&=\sum _{k=i_{start}}^{i_{end}}{\alpha }_{i,t}w_t\end{split}\) 
