A corrupted dataset is a more difficult version of the uncorrupted one.
Figure REF  shows the histogram of the memorization scores (computed from turn-over dropout) on uncorrupted MNIST and corrupted MNIST.
When there is no corruption, the scores follow a long tail distribution as described in [1]} (c.f. Figure REF ).
Corrupting an instance significantly increases its difficulty (memorization score): due to the random pattern/relationship between the image and the label, the model has no option but to memorize the instance.
Random examples in the corrupted scenario are the counterpart of the difficult examples in the uncorrupted scenario, as shown in Figure REF .
Hence, random data and real data are consistent with respect to memorization scores, and the methodology to study the properties of random data sheds light on difficult examples in the uncorrupted scenario.
