**Basics and Principles.** Instead of training on specific task goals, LLMs learn from unsupervised pre-training on large-scale text data. This is quite different from previous multi-task learning approaches, which aim to extend the training tasks as possible to achieve sufficient generalization. Thus, it is essential to reveal the basic principles or elements that establish the foundation of the abilities of LLMs. Although the basic idea of language models is intuitive, it is still challenging to formally explain why LLMs trained by simple language modeling objectives (_e.g._, next token prediction) can become capable of solving various real-world tasks. To investigate this problem, a promising approach is to study the capacity learning (or selection) mechanism based on unsupervised pre-training, since the model capacity of LLMs strongly depends on pre-training data. In addition, _scaling_ plays an important role in improving the capacity of LLMs [31, 55, 64], and it is very useful to conduct more theoretical analysis about how the behaviors of large models relate to those of small models, _e.g._, what behaviors of large models can be inferred from small models and what can't be predicted indeed. Another research direction is to explore more deep analysis on model generalization for LLMs, since increasing concerns have been raised about whether LLMs can generalize beyond the knowledge encoded by pre-training data. Furthermore, data contamination has become a severe issue for fairly assessing the performance of LLMs [738], and thus setting appropriate evaluation protocol will be the basis to investigate and analyze the model capacity of LLMs.