In this work, we study the underlying principles of transfer learning based upon an intriguing phenomenon has been recently discovered in terms of the representations, termed Neural Collapse (\(\mathcal {NC}\) ) [1]}, [2]}. Recently, for classification problems, the \(\mathcal {NC}\)  is an intriguing phenomenon has been discovered in terms of learned deep representations (see fig:NC), in which the last-layer features and classifiers collapse to simple but elegant mathematical structures on the training data: (i) for each class, the intra-class variability of last-layer features collapses to zero, and (ii) the between-class class means and the last-layer classifiers all collapse to the vertices of a Simplex Equiangular Tight Frame (ETF) up to scaling. This \(\mathcal {NC}\)  phenomenon [1]}, [2]} has been empirically demonstrated to persist across a variety of network architectures and datasets. Theoretically, recent works justified the prevalence of \(\mathcal {NC}\)  under simplified unconstrained feature models across a variety of training losses [5]}, [6]}, [7]}, [8]} and problem formulations [9]}, [10]}, [11]}. However, despite of recent endeavors of demystifying the \(\mathcal {NC}\)  phenomenon, a fundamental question lingers: is \(\mathcal {NC}\)  a blessing or a curse for deep representation learning? This work address this question in terms of transfer learning.
