The paper should not be considered as a push to expand model sizes and data sizes of language models beyond current scales. It is not guaranteed that new tipping points emerge in larger scales. Also, pre-training is not the only way to improve the performance of emergent abilities. For example, instruction tuning [46; 36; 6; 26] can improve the zero-shot performance of language models on unseen tasks, including the MMLU dataset. Future studies can analyze the acquisition of emergent abilities and lower the scale requirements.