Scaling lawsare functional relationships between the properties of interests (e.g., test loss or other performance metrics) and the scales of controllable factors regarding the optimization process or architecture (e.g., model sizes and numbers of training samples) (Villalobos, 2023). Along with the development of machine learning, characterizing scaling behaviors has garnered great research interest under the context of learning theories, bounding the generalization error given the number of training samples in the form of power laws (Vapnik and Chervonenkis, 1971; Valiant, 1984; Haussler, 1988; Amari et al., 1992). Nevertheless, overly strict assumptions hinder their practical applications. In recent years, statistical estimation on scaling gained fast progress for deep neural networks and spawns the introduction of scaling laws. Hestness et al. (2017) pioneers the trend and demonstrates power-law generalization error scaling across a breadth of factors but the power-law exponents differ from previous theoretical analysis. Kaplan et al. (2020); Hoffmann et al. (2022); Henighan et al. (2020) conduct more comprehensive investigations on Transformer architecture (Vaswani et al., 2017), further highlighting the power-law relationship on test loss regarding model sizes, the amount of training data and computation across orders of magnitudes. These findings foretell the performance gain with scaling quantitatively and guide the trade-off between larger models and more training data, directing to the later development of large language models (Brown et al., 2020; Hoffmann et al., 2022; OpenAI, 2023). Lately, progressive investigations propose amendments to existing scaling laws (Caballero et al., 2022; Alabdulmohsin et al., 2022), seeking theoretical explanations on the empirical formulas Bahri et al. (2021); Hutter (2021); Michaud et al. (2024), and exploring the functional relationships in broader scenarios (Hernandez et al., 2021; Frantar et al., 2023; Liu et al., 2023). The most relevant study to ours is Hashimoto (2021) which explores performance prediction under multiple data sources but is limited to small-scaled supervised learning tasks.