Online Similarity Learning for Big Data with Overfitting
In this paper, we propose a general model to address the overfitting problem in online similarity learning for big data, which is generally generated by two kinds of redundancies: 1) feature redundancy, that is there exists redundant (irrelevant) features in the training data; 2) rank redundancy, that is non redundant (or relevant) features lie in a low rank space. To overcome these, our model is designed to obtain a simple and robust metric matrix through detecting the redundant rows and columns in the metric matrix and constraining the remaining matrix to a low rank space. To reduce feature redundancy, we employ the group sparsity regularization, i.e. the <inline formula> <tex math notation="LaTeX"\ell _{2,1}/tex math><alternatives><inline graphic xlink:href="cong ieq1 2688360.gif" /alternatives>/inline formula> norm, to encourage a sparse feature set. To address rank redundancy, we adopt the low rank regularization, the <italic>m</italic>ax norm, instead of calculating the SVD as in traditional models using the nuclear norm. Therefore, our model can not only generate a low rank metric matrix to avoid overfitting, but also achieves feature selection simultaneously. For model optimization, an online algorithm based on the stochastic proximal method is derived to solve this problem efficiently with the complexity of <inline formula> <tex math notation="LaTeX"$O(d^2)/tex math><alternatives><inline graphic xlink:href="cong ieq2 2688360.gif" /alternatives>/inline formula> To validate the effectiveness and efficiency of our algorithms, we apply our model to online scene categorization and synthesized data and conduct experiments on various benchmark datasets with comparisons to several state of the art methods. Our model is as efficient as the fastest online similarity learning model OASIS, while performing generally as well as the accurate model OMLLR. Moreover, our model can exclude irrelevant redundant feature dimension simultaneously.
IEEE Transactions on Big Data