In this paper, we present a study on sample preselection in large training data set for CNN-based classification. To do so, we structure the input data set in a network representation, namely the Relative Neighbourhood Graph, and then extract some vectors of interest. The proposed preselection method is evaluated in the context of Handwritten Character Recognition, by using two data sets, up to several hundred thousands of images. It is shown that the graph-based preselection can reduce the training data set without degrading the recognition accuracy of a non pre-trained CNN shallow model.
The support-vector network is a new learning machine for two-group classification problems. The machine conceptually implements the following idea: input vectors are non-linearly mapped to a very high-dimension feature space. In this feature space a linear decision surface is constructed. Special properties of the decision surface ensures high generalization ability of the learning machine. The idea behind the support-vector network was previously implemented for the restricted case where the training data can be separated without errors. We here extend this result to non-separable training data. High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated. We also compare the performance of the support-vector network to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition. In dealing with large data sets, the reduced support vector machine (RSVM) was proposed for the practical objective to overcome some computational difficulties as well as to reduce the model complexity. In this paper, we study the RSVM from the viewpoint of sampling design, its robustness, and the spectral analysis of the reduced kernel. We consider the nonlinear separating surface as a mixture of kernels. Instead of a full model, the RSVM uses a reduced mixture with kernels sampled from certain candidate set. Our main results center on two major themes. One is the robustness of the random subset mixture model. The other is the spectral analysis of the reduced kernel. The robustness is judged by a few criteria as follows: 1) model variation measure; 2) model bias (deviation) between the reduced model and the full model; and 3) test power in distinguishing the reduced model from the full one. For the spectral analysis, we compare the eigenstructures of the full kernel matrix and the approximation kernel matrix. The approximation kernels are generated by uniform random subsets. The small discrepancies between them indicate that the approximation kernels can retain most of the relevant information for learning tasks in the full kernel. We focus on some statistical theory of the reduced set method mainly in the context of the RSVM. The use of a uniform random subset is not limited to the RSVM. This approach can act as a supplemental algorithm on top of a basic optimization algorithm, wherein the actual optimization takes place on the subset-approximated data. The statistical properties discussed in this paper are still valid Although a support vector machine (SVM) is one of the most frequently used classifiers in the field of intelligent transportation systems and shows competitive performances in various problems, it has the disadvantage of requiring relatively large computations in the testing phase. To make up for this weakness, diverse methods have been researched to reduce the number of support vectors determining the computations in the testing phase. This paper is intended to help engineers using the SVM to easily apply support vector number reduction to their own particular problems by providing a state-of-the-art survey and quantitatively comparing three implementations belonging to postpruning, which exploits the result of a standard SVM. In particular, this paper confirms that the support vector number of a pedestrian classifier using a histogram-of-oriented-gradient-based feature and a radial-basis-function-kernel-based SVM can be reduced by more than 99.5 without any accuracy degradation using iterative preimage addition, which can be downloaded from the Internet. A serious problem of support vector machine (SVM) is its low classifying speed. The speed depends on the number of support vectors. The clustering SVM, proposed in this paper, is a new method to reduce the number of support vectors. The method uses a k-means clustering technique to assign the data of each class to k groups, then we train the SVM based on a new dataset consist of the central vectors of each group. The value of k is the upper bound of the number of support vector in the class. Experiment results demonstrate that our method can control the tradeoff between the classifying speed and the performance of SVM.  