Grokking was first observed for algorithmic datasets by [1]}. Several formal or informal attempts have been made to understand grokking: (a) [2]} attributes grokking to the slow formation of good representations. (b)  [3]} suggests that
generalizable solutions achieve lower loss than overfitting solutions, providing a training signal
encouraging generalization. (c) [4]} suggests grokking is a phase change due to limited data and regularization. (d) [5]} suggests that generalization is due not to random search, but to hidden progress of SGD to gradually amplify a Fourier gap. (e) [6]} links grokking to the "Slingshot mechanism" specific to adaptive optimizers. (f)  [7]} describes training as a random walk over parameters. Our conclusion supports (a)(b)(c)(d), but does not necessarily negate (e)(f).
