In earlier work, people mainly used the "bag of words" approach in algorithms such as Naive Bayes, Decision Tree, and SVM. However, recent studies (Peters et al., 2018; Radford et al., 2018; <cite>Devlin et al., 2018)</cite> showed that contextual word embeddings perform quite better than traditional word embeddings in many different NLP tasks as a result of their superior capacity of meaning representation.