The GPT family is one of the most common examples of AutoRegressive language models. It was first proposed by Radford et al. (2018), who identified a basic architecture of unsupervised pre-training followed by fine-tuning. Radford et al. (2019) later proposed zero-shot learning based on GPT. Later, Brown et al. (2020) proposed GPT-3 using an approach similar to Radford et al. (2019), which involved scaling up and abandoning fine-tuning. They also utilized alternately dense and locally banded sparse attentional patterns in the transformer layer, similar to the sparse transformer Child et al. (2019). There are also several related studies, such as GPT-NEO(Black et al., 2022) and ChatGPT, which use Reinforcement Learning from Human Feedback (RLHF). RLHF has significantly enhanced the accuracy of GPT models. Although ChatGPT is not open source, many researchers Jiang et al. (2023) (Khattab et al., 2022) still use its API for generative tasks in RALM. Recently, report on GPT-4(Achiam et al., 2023) have appeared, with the main focus on building a predictable and scalable deep learning stack dedicated to improving GPT-4's safety and alignment. Many researchers Asai et al. (2023) (Luo et al., 2023) have recently used GPT-4 to generate prompts for RALM.