(_i.e._, following the scaling law [30]). A number of studies have explored the performance limit by training an ever larger PLM (_e.g.,_ the 175B-parameter GPT-3 and the 540B-parameter PaLM). Although scaling is mainly conducted in model size (with similar architectures and pre-training tasks), these large-sized PLMs display different behaviors from smaller PLMs (_e.g.,_ 330M-parameter BERT and 1.5B-parameter GPT-2) and show surprising abilities (called _emergent abilities_[31]) in solving a series of complex tasks. For example, GPT-3 can solve few-shot tasks through _in-context learning_, whereas GPT-2 cannot do well. Thus, the research community coins the term _"large language models (LLM)"1_ for these large-sized PLMs [32, 33, 34, 35], which attract increasing research attention (See Figure 1). A remarkable application of LLMs is _ChatGPT2_ that adapts the LLMs from the GPT series for dialogue, which presents an amazing conversation ability with humans. We can observe a sharp increase of the arXiv papers that are related to LLMs after the release of ChatGPT in Figure 1.