A wide range of studies have shown that the performance of a language model exhibits a notable growth pattern as the number of parameters and data size increase, following a power-law relationship [17, 20, 21, 22, 23, 24]. This scaling law plays a fundamental role in the development of large language models, enabling us to estimate optimal configurations of large models from the training logs of much smaller models [22, 23]. As mentioned in the GPT-4 technical report [1], some aspects of GPT-4's performance can be accurately predicted based on models trained with no more than 1/1,000th the compute of GPT-4. By properly utilizing the scaling law, we avoid the need to perform extensive model-specific tuning on large models.