where the expectation is taken with respect to the sample \(S_{n}\) , the notation \(\lesssim \)  suppresses absolute multiplicative constants, and the optimal risk \(R(\omega ^{*})\)  is equal to the variance of the noise random variable \(\xi \) .
Remarkably, the bound (REF ) depends neither on the exact form of the covariance matrix \(\Sigma \)  nor on the magnitude of \(\omega ^{*}\) .
Recent work [1]} shows that if the model is well-specified then for any distribution of the covariates \(X\)  such that the sample covariance matrix is almost surely invertible and any \(n \ge d\) , the excess risk of unconstrained least squares is exactly equal to the minimax risk. While the above result attests to the existence of regimes where least squares is a statistically optimal estimator in a minimax sense, there is a growing interest in the statistics and machine learning communities in understanding the robustness of statistical estimators to various forms of model misspecification. For instance, the regression function \(\operatorname{\mathbb {E}}(Y \mid X = \cdot )\)  might be non-linear, or the distribution of the noise random variable \(\xi \)  might depend on the corresponding covariate \(X\) .
