Over a balanced training dataset \({\mathbf {x}_{k,i},\mathbf {y}_k}\)  with \(n = n_1 = n_2= \cdots = n_K\) , it has been widely observed that last-layer features \(\mathbf {H}\)  and classifiers \(\mathbf {W}\)  of a trained network exhibit simple but elegant mathematical structures [1]}, [2]}, that we highlight two key properties below.Additionally, self-duality convergence has also been observed in the sense that \(\mathbf {w}_k = c^{\prime } \overline{\mathbf {h}}_k \)  for some \(c^{\prime }>0\) , but this is not the main focus of this work.
