Ever since the Turing Test was proposed in the 1950s, humans have explored the mastering of language intelligence by machine. Language is essentially a complex, intricate system of human expressions governed by grammatical rules. It poses a significant challenge to develop capable artificial intelligence (AI) algorithms for comprehending and grasping a language. As a major approach, _language modeling_ has been widely studied for language understanding and generation in the past two decades, evolving from statistical language models to neural language models. Recently, pre-trained language models (PLMs) have been proposed by pre-training Transformer models over large-scale corpora, showing strong capabilities in solving various natural language processing (NLP) tasks. Since the researchers have found that model scaling can lead to an improved model capacity, they further investigate the scaling effect by increasing the parameter scale to an even larger size. Interestingly, when the parameter scale exceeds a certain level, these enlarged language models not only achieve a significant performance improvement, but also exhibit some special abilities (_e.g.,_ in-context learning) that are not present in small-scale language models (_e.g.,_ BERT). To discriminate the language models in different parameter scales, the research community has coined the term _large language models (LLM)_ for the PLMs of significant size (_e.g.,_ containing tens or hundreds of billions of parameters). Recently, the research on LIMs has been largely advanced by both academia and industry, and a remarkable progress is the launch of ChatGPT (a powerful AI chatbot developed based on LIMs), which has attracted widespread attention from society. The technical evolution of LIMs has been making an important impact on the entire AI community, which would revolutionize the way how we develop and use AI algorithms. Considering this rapid technical progress, in this survey, we review the recent advances of LIMs by introducing the background, key findings, and mainstream techniques. In particular, we focus on four major aspects of LIMs, namely pre-training, adaptation tuning, utilization, and capacity evaluation. Furthermore, we also summarize the available resources for developing LIMs and discuss the remaining issues for future directions. This survey provides an up-to-date review of the literature on LIMs, which can be a useful resource for both researchers and engineers.