
CRF\(^{\otimes }\) (GloVe):

This uses 300-dimensional GloVe embeddings [1]} combined with the CRF\(^{\otimes }\) .


CRF\(^{\otimes }\) (GloVe, biLSTM)

This uses 300-dimensional GloVe embeddings combined with a biLSTM and the CRF\(^{\otimes }\) . The forward and backward LSTM states each have 300 units.


CRF\(^{\otimes }\) (BERT)

This uses the final (768-dimensional) layer of the BERT-Base model [2]} with the CRF\(^{\otimes }\) . BERT embeddings are defined on sub-word tokens rather than on words. Therefore if the BERT tokenizer splits a word into multiple sub-word tokens, we take the mean of the sub-word embeddings as the word embedding.


CRF\(^{\otimes }\) (Flair)

This uses Flair embeddings [3]} with the CRF\(^{\otimes }\) . [3]} recommend concatenating 100-dimensional GloVe embeddings to their forward and backward language model embeddings. The resulting Flair embeddings are 4196-dimensional.


