We observed a performance improvement over competitive baselines such as FastText (FT) (Joulin et al., 2017) and Deepmoji <cite>(Felbo et al., 2017)</cite> , which is most noticeable in the case of infrequent emojis.