Recently, contextual word embeddings such as those provided by pre-trained language models have become more prominent [1]}, [2]}, [3]}, [4]}. Contextual embeddings incorporate sentence-level information, therefore they can be used directly with linear chain CRFs to achieve state of the art performance on sequence labeling tasks [5]}, [6]}.
