\(\bullet\)_GPT-1_. In 2017, the Transformer model [22] was introduced by Google, and the OpenAI team quickly adapted their language modeling work to this new neural network architecture. They released the first GPT model in 2018, _i.e._, GPT-1 [122], and coined the abbreviation term _GPT_ as the model name, standing for _Generative Pre-Training_. GPT-1 was developed based on a generative, decoder-only Transformer architecture, and adopted a hybrid approach of unsupervised pretraining and supervised fine-tuning. GPT-1 has set up the core architecture for the GPT-series models and established the underlying principle to model natural language text, _i.e._, predicting the next word.