Other models have been released these last two years, such as BERT [1]} (trained on English data then multi-lingual data) or Ernie [2]}, which are language and context dependent. These models need a lot of data to be trained, such as the entire Wikipedia totalling over 2,500 million words and Book Corpus totalling over 800 million words.
