Traditional theory on overparameterization suggests that scaling up model size in sufficiently trained models can enhance memorization of training data [6], improve generalization error [15, 27, 28], and better fit complex target functions [5, 23]. However, these results often overlook large constant or polynomial factors, leading to a significant discrepancy from practical outcomes.