Molecules We search for grokking using the graph convolutional neural network (GCNN) for QM9 dataset [1]}. Similar to Eq. (REF ), we define the reduced training/test losses, which are only dependent on weight norm \(w\)  and data size \(N\) . As shown in Figure REF (a)(b), when data size is large, training and test losses have similar "U" shapes, hence grokking is impossible via the "LU mechanism". When data size is small, training and test losses mismatch somewhere in the region \(\alpha =w/w_0>1\) , making grokking possible. Indeed, shown in Figure REF (d), there is a sharp drop in test loss around \(10^4\)  steps if initialization is 3 times larger than standard, while standard initialization does not lead to grokking. Note that zero weight decay is applied in both cases, implying the existence of implicit regularizations.
