Soft Kernel Target Alignment for Two Stage Multiple Kernel Learning
The two stage multiple kernel learning (MKL) algorithms gained the popularity due to their simplicity and modularity. In this paper, we focus on two recently proposed two stage MKL algorithms: ALIGNF and TSMKL. We first show through a simple vectorization of the input and target kernels that ALIGNF corresponds to a non negative least squares and TSMKL to a non negative SVM in the transformed space. Then we propose ALIGNF+ a soft version of ALIGNF, based on the observation that the dual problem of ALIGNF is essentially a one class SVM problem. It turns out that the ALIGNF+ just requires an upper bound on the kernel weights of original ALIGNF. This upper bound makes ALIGNF+ interpolate between ALIGNF and the uniform combination of kernels. Our experiments demonstrate favorable performance and improved robustness of ALIGNF+ comparing to ALIGNF. Experiments data and code written in python are freely available at github (https:/github.com/aalto ics kepaco/softALIGNF)
DS