where \(\ell (\cdot , \cdot )\)  is the loss function and \(\otimes \)  is the tensor product.
Liang et al. [1]} introduce the geometric invariance property that a complexity measure should satisfy in order to study generalization capability.
The invariance property essentially says that, since many different continuous operations may lead to exactly the same prediction, thus the generalization should only depend on the equivalence classes obtained by these continuous transformations. Specific parameterization should not affect the generalization. The complexity measure used to investigate generalization should satisfy the invariance property. They demonstrate that this Fisher-Rao norm satisfies the invariance property. Let \(\theta _1, \theta _2\)  denote two parameter settings of a model \(f\) . If \(f_{\theta _1} = f_{\theta _2}\) , then their Fisher-Rao norms are equal, that is, \(||\theta _1||_{fr} = ||\theta _2||_{fr}\) . In particular, the Fisher-Rao norm remains invariant during the node-wise rescaling of a model.
