Implicit Bias of First-order Optimization Methods. [1]} proved that gradient descent on linear logistic regression with separable data converges in the direction of the max \(L^2\)  margin solution of the corresponding hard-margin Support Vector Machine, and motivate a line of works on the bias of GD on linear model. For example, [2]} studied other loss with different tails; [3]} proved similar result without assuming separable data; [4]} worked on the problem in the adversarial scenario; [5]} analyzed the case when there is only a ReLU neuron.
