On the other hand, in the classification setting, the implicit bias analysis becomes significantly more challenging, and several questions remain open despite significant progress in the past few years. A key differentiating factor in the classification setting is that the loss function does not attain its minimum at a finite value, and the weights have to grow to infinity.
It has been shown that for the logistics loss, the gradient descent iterates converge to the \(\ell _2\) -maximum margin SVM solution in direction [1]}, [2]}.
However, such characterizations for mirror descent are missing in the literature.
Because it is possible for optimization algorithms to exhibit implicit bias in regression but not in classification (and vice versa) [3]}, resolving this gap of knowledge warrants careful analysis.
See Table REF  for a summary.
