Centred Kernel Alignment (\(\mathbb {CKA}\) ) proposed by [1]} is a metric to measure the representation similarity between two networks.
Here in this work, we leverage the linear CKA (as later shown in Eq. REF ) to evaluate the similarity between the feature representations (obtained by the feature extractor of our model) learnt by our models in different stages, in order to observe how much the network is changed/updated across stages. We denote the feature extractor in our learnt model after finishing the training stage \(t\)  as \(\mathcal {F}_t\) , the data available or used in the training stage \(\dot{t}\)  as \(D_{\dot{t}}\) , and the collection of feature representations of \(D_{\dot{t}}\)  projected by \(\mathcal {F}_t\)  as \(R_{t,\dot{t}} \in \mathbb {R}^{n \times d}\)  where \(n\)  and \(d\)  denote the number of data samples in \(D_{\dot{t}}\)  and the feature dimensionality respectively. Given the data samples \(D_{\dot{t}}\) , the representation similarity between two stages \(t\)  and \(t+1\)  is now formulated as
\( \mathbb {CKA}(R_{t,\dot{t}}, R_{t+1,\dot{t}})=\frac{\Vert R_{t,\dot{t}}^\top R_{t+1,\dot{t}}\Vert ^2_\text{F}}{\Vert R_{t,\dot{t}}^\top R_{t,\dot{t}}\Vert _\text{F} \Vert R_{t+1,\dot{t}}^\top R_{t+1,\dot{t}}\Vert _\text{F}}\) 
