The backbone model, illustrated in Figure REF , is a BERT-CRF model [1]}, [2]}. It takes advantage of large scale pre-trained language models—using BERT-based encoder to create contextual representations for each token, and a probabilistic graphical model—using conditional random fields [3]} to capture dependencies between neighbouring tags.
<FIGURE>