Double descent is the phenomenon that performance first gets worse and then gets better as we increase the model size, data size, training epochs or regularization [1]}, [2]}. The typical "U" shape of test loss in this paper does not conflict with double descent, because we are plotting the weight norm instead of the number of model parameters [3]}. However, the "U"-shape should better be considered as empirically common rather than provably universal. In fact, the interaction between properties of data and inductive biases of learning algorithms can be more complicated than double descent [4]}, [5]}.
