\begin{table}
\begin{tabular}{|c|c|c|c|c|c|c|} \hline Parameter & \(\alpha_{N}\) & \(\alpha_{S}\) & \(\alpha_{B}\) & \(N_{c}\) & \(S_{c}\) & \(B_{*}\) \\ \hline \hline C4 Value & \(0.076\) & \(0.67\) & \(0.205\) & \(1.5\times 10^{14}\) & \(2.6\times 10^{3}\) & \(1.7\times 10^{8}\) \\ \hline
**[dataset as the test set and performed a deduplication process to ensure the removal of any overlapping text from the training set. In total, we trained only 10 small models, each with a maximum of 60M parameters, to estimate the constant terms of the scaling-law formulas. The estimated constant terms are presented in Table 1. The actual and predicted loss trajectories of a 2B model (30 times larger than the small model used to estimate the constant terms) using the estimated formulas are depicted on the left of Figure 1. It can be observed that the predicted loss trajectory closely aligns with the actual loss trajectory. Our estimated constant terms are also not far from those estimated in [10] despite using different setups, possibly due to the similar distributions of C4 and WebText, both of which consist of crawled website text. This reinforces the assertion in [11] that the power term in scaling laws primarily relies on the data manifold.