One of the most intriguing and remarkable recent mathematical discoveries in deep learning is the constancy of the so-called Neural Tangent Kernel (NTK) for certain wide neural networks due to Jacot, Gabriel and Hongler [1]}. As the width of certain networks increases to infinity, they undergo transition to linearity (using the term and following the discussion in [2]}) and become linear functions of their parameters. Specifically, consider a model \(f({\mathbf {w}},{\mathbf {x}})\) , where the vector \({\mathbf {w}}\in {\mathbb {R}}^M\)  represents trainable parameters. The tangent kernel at \({\mathbf {w}}\) , associated to \(f\)  is defined as follows:
\(K_{({\mathbf {x}},{\mathbf {z}})}({\mathbf {w}}) := \langle \nabla _{\mathbf {w}}f({\mathbf {w}};{\mathbf {x}}),\nabla _{\mathbf {w}}f({\mathbf {w}};{\mathbf {z}})\rangle , \quad \textrm {for fixed inputs }{\mathbf {x}}, {\mathbf {z}}\in \mathbb {R}^{d}.\) 
