In contrast to supervised machine learning that optimises model using labelled data only, semi-supervised learning aims to make use of both labelled data and unlabelled data. Pre-training language representation models on unlabelled data and then adapting pre-trained model to the downstream supervised task is one type of semi-supervised learning. It has demonstrated its effectiveness in NLP during the past decade [1]}, [2]}, [3]}, [4]}, [5]}. In this section, we briefly describe the design in Flair and more options are discussed in Section REF .
