Notably, it has been observed that even in the absence of any explicit regularization, the interpolating solutions obtained by the standard gradient-based optimization algorithms, such as (stochastic) gradient descent, tend to generalize well. Recent research has highlighted that such optimization algorithms favor particular types of solutions, i.e., they implicitly regularize the learned models.
Importantly, such implicit biases are shown to play a crucial role in determining generalization performance,
e.g.,  [1]}, [2]}, [3]}.
