Finally, we highlight that the construction used to prove our main lower bound (REF ) is specifically designed to simultaneously satisfy moment equivalence assumptions (on the noise and the design) with ill-behaved constants. The fact that non-linear estimators can always achieve the optimal \(d/n\)  rate shows that they are insensitive to moment equivalence constants. Thus, the observed statistical gap between proper and improper estimators is of interest to the line of work investigating estimators (typically proper) robust to heavy-tailed data. Indeed, following the first appearance of the preprint of this work, this observation has motivated the design of a non-linear statistical estimator robust to heavy-tailed data that works without any assumptions on the distribution of the covariates [1]}.
