Soft-SVM Regression For Binary Classification
The binomial deviance and the SVM hinge loss functions are two of the most
widely used loss functions in machine learning. While there are many
similarities between them, they also have their own strengths when dealing with
different types of data. In this work, we introduce a new exponential family
based on a convex relaxation of the hinge loss function using softness and
class-separation parameters. This new family, denoted Soft-SVM, allows us to
prescribe a generalized linear model that effectively bridges between logistic
regression and SVM classification. This new model is interpretable and avoids
data separability issues, attaining good fitting and predictive performance by
automatically adjusting for data label separability via the softness parameter.
These results are confirmed empirically through simulations and case studies as
we compare regularized logistic, SVM, and Soft-SVM regressions and conclude
that the proposed model performs well in terms of both classification and
prediction errors.