Comparison.  The \(p=2\)  result of the inner product kernel is related to [1]} who derived generalization bounds for the minimum RKHS norm estimator. They showed that when the data covariance matrix and the kernel matrix enjoy certain decay of the eigenvalues, the generalization bound vanishes as \(n\)  goes to infinity. For example, for exponential kernel and the covariance matrix \(\Sigma := \mathbb {E} [x x^\top ]\)  with the \(j\) -th eigenvalue \(\lambda _j (\Sigma ) = j^{- \alpha }\) , the \(\ell _2\)  generalization bound becomes \(\mathcal {O} (n^{- \frac{\alpha }{2 \alpha + 1}})\)  when \(\alpha \in (0, 1)\)  and \(n > d\) . In comparison, we get an optimal generalization bound \({\mathcal {O}} \left(n^{- 1 / 2}\right)\)  for identity covariance matrix on a globally Lipschitz loss function. Again, we emphasize that the i.i.d. assumption can be relaxed if more new results of random matrix theory are available. Further, we extend the works in [1]} by proving a new result of the RBF kernel. Note that the result of the RBF kernel is not under the high-dimensional setting; thus it is not a direct adaptation of [1]}, and the proof itself is of independent interest.
