Pre-trained self-supervised language models [1]}, [2]}, [3]} have improved downstream NLP tasks' performance. Most state-of-the-art methods use pretrained models as contextualized token features but do not alter the underlying model's structure. In contrast, we experiment with the augmentation of a pretrained transformer model to address the nested entity recognition task.
