Like in fastText [4] , ELMo <cite>[33]</cite> breaks the tradition of word embeddings by incorporating sub-word units, but ELMo <cite>[33]</cite> has also some fundamental differences with previous shallow representations such as fastText or Word2Vec.