The second question, discussed by Doersch  [1]}, is the so-called supervision collapse.
This phenomenon is inherent to pre-training with a fixed set of labels:
the network learns to focus on the mapping between images and the labels of the pre-training stage, but can discard information that is relevant to other downstream tasks.
In other terms, pre-training on large-scale classification datasets does not necessarily align with the goal of learning general-purpose features, as it uses only a subset of the available information controlled by the given dataset categorization bias [2]}.
