\(\bullet\)_Neural language models (NLM)_. NLMs [17, 18, 1] characterize the probability of word sequences by neural networks, _e.g.,_ multi-layer perceptron (MLP) and recurrent neural networks (RNNs). As a remarkable contribution, the work in [1] introduced the concept of _distributed representation_ of words and built the word prediction function conditioned on the aggregated context features (_i.e.,_ the distributed word vectors). By extending the idea of learning effective features for text data, a general neural network approachwas developed to build a unified, end-to-end solution for various NLP tasks [2]. Furthermore, word2vec [19, 20] was proposed to build a simplified shallow neural network for learning distributed word representations, which were demonstrated to be very effective across a variety of NLP tasks. These studies have initiated the use of language models for representation learning (beyond word sequence modeling), having an important impact on the field of NLP.