Recipe for Fast Large-scale SVM Training: Polishing, Parallelism, and more RAM!
Support vector machines (SVMs) are a standard method in the machine learning
toolbox, in particular for tabular data. Non-linear kernel SVMs often deliver
highly accurate predictors, however, at the cost of long training times. That
problem is aggravated by the exponential growth of data volumes over time. It
was tackled in the past mainly by two types of techniques: approximate solvers,
and parallel GPU implementations. In this work, we combine both approaches to
design an extremely fast dual SVM solver. We fully exploit the capabilities of
modern compute servers: many-core architectures, multiple high-end GPUs, and
large random access memory. On such a machine, we train a large-margin
classifier on the ImageNet data set in 24 minutes.