In this paper, we present a study on sample preselection in large training data set for CNN-based classification. To do so, we structure the input data set in a network representation, namely the Relative Neighbourhood Graph, and then extract some vectors of interest. The proposed preselection method is evaluated in the context of Handwritten Character Recognition, by using two data sets, up to several hundred thousands of images. It is shown that the graph-based preselection can reduce the training data set without degrading the recognition accuracy of a non pre-trained CNN shallow model.
Among the proposed techniques, that falls into the aforementioned categories, it is worth mentioning the usage of: (i) random selection techniques @cite_7 , (ii) clustering techniques @cite_3 or graph-based techniques @cite_0 . One can refer to thorough surveys that have been done recently by @cite_6 in 2012 (for NN based classification), and by @cite_1 in 2014 for (Support Vector Machine (SVM) @cite_4 based classification).
@cite_4 @cite_7 @cite_1 @cite_3 @cite_6 @cite_0
2128529555 2164617909 2130828936 2151245131
Linear support vector machines (SVM) are useful for classifying large-scale sparse data. Problems with sparse features are common in applications such as document classification and natural language processing. In this paper, we propose a novel coordinate descent algorithm for training linear SVM with the L2-loss function. At each step, the proposed method minimizes a one-variable sub-problem while fixing other variables. The sub-problem is solved by Newton steps with the line search technique. The procedure globally converges at the linear rate. As each sub-problem involves only values of a corresponding feature, the proposed approach is suitable when accessing a feature is more convenient than accessing an instance. Experiments show that our method is more efficient and stable than state of the art methods such as Pegasos and TRON. In this paper, we first develop a direct Bayesian based support vector machine by combining the Bayesian analysis with the SVM. Unlike traditional SVM-based face recognition method that needs to train a large number of SVMs, the direct Bayesian SVM needs only one SVM trained to classify the face difference between intra-personal variation and extra-personal variation. However, the added simplicity means that the method has to separate two complex subspaces by one hyper-plane thus affects the recognition accuracy. In order to improve the recognition performance we develop three more Bayesian based SVMs, including the one-versus-all method, the hierarchical agglomerative clustering based method, and the adaptive clustering method. We show the improvement of the new algorithms over traditional subspace methods through experiments on two face databases, the FERET database and the XM2VTS database. A serious problem of support vector machine (SVM) is its low classifying speed. The speed depends on the number of support vectors. The clustering SVM, proposed in this paper, is a new method to reduce the number of support vectors. The method uses a k-means clustering technique to assign the data of each class to k groups, then we train the SVM based on a new dataset consist of the central vectors of each group. The value of k is the upper bound of the number of support vector in the class. Experiment results demonstrate that our method can control the tradeoff between the classifying speed and the performance of SVM. We propose a pre-selection method for training support vector machines (SVM) with a large-scale dataset. Specifically, the proposed method selects patterns around the class boundary and the selected data is fed to train an SVM. For the selection, that is, searching for boundary patterns, we utilize a relative neighborhood graph (RNG). An RNG has an edge for each pair of neighboring patterns and thus, we can find boundary patterns by looking for edges connecting patterns from different classes. Through large-scale handwritten digit pattern recognition experiments, we show that the proposed pre-selection method accelerates SVM training process 5–15 times faster without degrading recognition accuracy.