Regarding the terminology, we note the word pairs proper and linear, as well as improper and non-linear are used synonymously in this work. Thus, the separation between proper and improper learning is not to be confused with the separation between linear and non-linear procedures considered in other works. For example, the work [1]} finds a statistical gap between minimax rates achievable by procedures that are linear and non-linear in the observed response variables. In contrast, in our case, the linearity/properness means that an estimator always selects a predictor of the form \(x \mapsto \langle x, w\rangle \)  for some \(w \in \mathcal {W}_{b}\) , while non-linear/improper predictor is allowed to select predictors that are not of the above form.
