A mention's immediate context is a proxy for its type. Here, the mention's immediate context is a sequence of tokens where the mention \({\rm m}_{ij}\)  is replaced with a single [MASK] token. Then, we represent the immediate entity context by extracting the upper most layer representation of pre-trained BERT [1]} corresponding to the [MASK] token.
\(\mathbf {c}_{ij} = {\rm BERT}(\lbrace {\rm lctx}_{ij}, {\rm [MASK]}, {\rm rctx}_{ij}\rbrace ) \) 
