There are also approaches that investigate the impact of more complex models like RNN and LSTM. [1]} created a hierarchical language model that uses RNN to combine morphemes of a word to obtain a word representation. Their model performed well on word similarity task. Similarly, [2]} create Char2Vec BiLSTM for embedding words and train a language model with SG objective. Their model excels at the syntactic similarity.
