A commonly-used method to study memorization is to corrupt the training data by replacing part of true labels with random labels.
With this method, previous work [1]}, [2]}, [3]} focuses on comparing the differences between the scenario of label corruption and the scenario of no corruption to understand memorization.
On the contrary, we juxtapose the two scenarios and focus on comparing the easy and difficult examples to find what is in common regarding optimization.
<FIGURE>