There have been several approaches to directly incorporate contextual information into sequence labeling architectures. The Conv-CRF [1]}, biLSTM-CRF [2]}, and biLSTM-CNN-CRF [3]} each augment the CRF with either a convolutional network, a bidirectional LSTM, or both in order to use contextual features when labeling a given word. More recently, [4]} propose an alternative LSTM structure for encoding text which consists of a parallel state for each word, achieving improved results over the biLSTM-CRF for sequence labeling tasks. GCDT [5]} improves an RNN-based architecture by augmenting it with a sentence-level representation which captures wider contextual information. [6]} learn representations encoding both sentence-level and document-level features for named entity recognition using a hierarchical bidirectional LSTM. [7]} leverage the fact that BERT can represent inputs consisting of several sentences in order to use cross-sentence context when performing named entity recognition.
