Pre-trained language representations.
Recent researches showed that language models generically trained on massive corpus is beneficial to various specific NLP tasks [1]}, [2]}. Language representation has been an active area of research for years. Tons of effective approaches have been developed from feature-based approaches [3]}, [4]}, [5]} to fine-tuning approaches [6]}, [7]}, [2]}.
<FIGURE>