We handle entity mentions in the form of tags assigned to each token with the conventional BIO or BIOUL formats [1]}, [2]}, by embedding each tag into a multidimensional vector space. We then sum the tag embeddings of different entities (depths) at the same position with each other.
The sentences are tokenized and represented with the contextual embedding, Transformer model BERT (for English documents [3]}) or CamemBERT (for French [4]}). The word embedding and the tag embeddings from previous iterations are fed into a linear CRF [5]} predicting flat entity mentions (Figure REF a).
