Fig. 2: An evolution process of the four generations of language models (LM) from the perspective of task solving capacity. Note that the time period for each stage may not be very accurate, and we set the time mainly according to the publish date of the most representative studies at each stage. For neural language models, we abbreviate the paper titles of two representative studies to name the two approaches: NPLM [1] (_“A neural probabilistic language model”_) and NLPS [2] (_“Natural language processing (almost) from scratch”_). Due to the space limitation, we don’t list all representative studies in this figure.