Emojis are ideograms which are naturally combined with plain text to visually complement or condense the meaning of a message. Despite being widely used in social media, their underlying semantics have received little attention from a Natural Language Processing standpoint. In this paper, we investigate the relation between words and emojis, studying the novel task of predicting which emojis are evoked by text-based tweet messages. We train several models based on Long Short-Term Memory networks (LSTMs) in this task. Our experimental results show that our neural model outperforms two baselines as well as humans solving the same task, suggesting that computational models are able to better capture the underlying semantics of emojis.
The most similar model is by , which attempts to predict the hashtag given a tweet. This is a similar task with a similar motivation towards better language understanding. They present a character composition model for tweets. The main difference with their work is that they encode the entire tweet as a sequence of characters with bidirectional GRUs @cite_21 while we encode each token as a sequence of characters with bidirectional LSTMs obtaining embeddings of tokens, and each tweet as a bidirectional sequence (again with LSTMs) of word embeddings avoiding sparsity and with the potentiality of being applied to longer sequences. It is to be noted that our model can also be used as a way of getting pretrained embeddings of tweets, by extracting the learned vectors from our bidirectional LSTMs.
@cite_21
1924770834
In this paper we compare different types of recurrent units in recurrent neural networks (RNNs). Especially, we focus on more sophisticated units that implement a gating mechanism, such as a long short-term memory (LSTM) unit and a recently proposed gated recurrent unit (GRU). We evaluate these recurrent units on the tasks of polyphonic music modeling and speech signal modeling. Our experiments revealed that these advanced recurrent units are indeed better than more traditional recurrent units such as tanh units. Also, we found GRU to be comparable to LSTM.