**How Emergent Abilities Relate to Scaling Laws**. In existing literature [30, 31, 34], scaling laws and emergent abilities provide two perspectives to understand the advantage of large models over small models. In general, scaling law (often measured by _language modeling loss_) describes predictable performance relation with the potential effect of diminishing returns, while emergent abilities (often measured by _task performance_) are unpredictable but very profitable once such abilities actually emerge. Since the two perspectives reflect different performance trends (continuous improvement _v.s._ sharp performance leap), they might lead to misaligned findings or observations. There are also extensive debates on the rationality of emergent abilities. A popular speculation is that emergent abilities might be partially attributed to the evaluation setting for special tasks (_e.g.,_ the discontinuous evaluation metrics) [70, 71]: when evaluation metrics are altered accordingly, the sharpness of the emergent ability curve would disappear. However, the performance of LLMs on most tasks are perceived by users naturally in a discontinuous way. For instance, end users prefer a reliable code generated by LLMs that can successfully pass the test case, but are less interested in selecting a better code with fewer errors between two failed ones. More recently, a study [72] proposes a new evaluation setting that can enlarge the resolution of task metrics, making task performance more predictable. Despite these efforts, more fundamental research (_e.g.,_ grokking10) about the working mechanism of LLMs is still in need to understand the emergence of certain abilities. The subtle relation between scaling law and emergent abilities can be explained by analogy with the ability acquisition of human11. Take the speaking ability as an example. For children, language development (especially infants) can be also considered as a multi-level process where "emergent abilities" occur. Specially, the language ability would relatively stable within a time interval, but qualitative change only occurs when evolving into another ability level (_e.g.,_ from speaking simple words to speaking simple sentences). Such a learning process is essentially not _smooth_ and _stable_ (_i.e.,_ language ability does not develop at a constant rate over time), though a child actually grows every day. It is interesting that young parents would be often surprised by unexpected progress of the speaking ability exhibited by their babies.