Context LengthAs mentioned, the context length significantly influences the values of the constant terms in scaling-law formulas. Anchoring all constant terms to a specific context length means that we need to rerun the estimation process for every new context length, which is rather inefficient because it is common to adjust the context length to fit various tasks. Given that the loss value at each position also approximately follows a power-law relation [13], it would be possible to include the context length directly as a parameter of the formulas.