Below we try to answer the question from the perspective of optimization (SGD updates) of real data and random dataWe use Backpack [1]} to extract the gradient of each example in the batch during the backward pass..
Figure REF  shows the evolution of the average cosine similarity (Equation (REF )) and the average contribution (Equation (REF )) of easy examples and difficult examples.
<FIGURE>