Using the notion of average stability, we prove a tight excess risk upper bound for the ridge regression estimator (see [1]}, [2]}, [3]} for a detailed account of stability in our context). The novel ingredient in our proof is the exploitation of the curvature of the squared loss in the stability-fitting trade-off. As a result, we show that the ridge estimator does not suffer from an excess factor \(\log (\min \lbrace n, d\rbrace )\)  that appears in an upper bound on the localized quadratic process. Moreover, we demonstrate that this logarithmic term is unavoidable for least squares in some regimes, thus showing an interesting performance gap between constrained and penalized least squares.
