Constrained ERM with the squared loss is also actively studied in an on-going line of work concerning the shape restricted regression literature (e.g., [1]}, [2]}, [3]}, [4]}), where the least squares projection is performed over constraint sets that may be significantly more complex than Euclidean balls. In particular, when considering some expressive nonparametric classes of functions, ERM can be either optimal [4]} or suboptimal [6]}, [7]}, depending on some additional properties of these classes. In contrast, our results establish suboptimality of the constrained least squares estimator for a parametric class that has a small intrinsic complexity. The work [1]} allows more general convex constraints and shows that ERM can be rate suboptimal. However, establishing suboptimality of ERM in our setting is more complicated: we are not free to choose an arbitrary ill-behaved convex constraint set and also, we study a random design setting and thus cannot choose a fixed set of ill-behaved covariates \(X_i\) . We additionally refer to [9]} for an extensive discussion on optimality and suboptimality of least squares and maximum likelihood estimators in different setups.
