Large datasets have been crucial to the success of deep learning models in the recent years, which keep performing better as they are trained with more labelled data. While there have been sustained efforts to make these models more data-efficient, the potential benefit of understanding the data itself, is largely untapped. Specifically, focusing on object recognition tasks, we wonder if for common benchmark datasets we can do better than random subsets of the data and find a subset that can generalize on par with the full dataset when trained on. To our knowledge, this is the first result that can find notable redundancies in CIFAR-10 and ImageNet datasets (at least 10 ). Interestingly, we observe semantic correlations between required and redundant images. We hope that our findings can motivate further research into identifying additional redundancies and exploiting them for more efficient training or data-collection.
An early mention of trying to reduce the training dataset size can be seen in @cite_28 . Their proposed algorithm splits the training dataset into many smaller training sets and iteratively removes these smaller sets until the generalization performance falls below an acceptable threshold. However, the algorithm relies on creating many small sets out of the given training set, rendering it impractical for modern usage.
@cite_28
209945127
In the standard support vector machines for classification, training sets with uneven class sizes results in classification biases towards the class with the large training size. That is to say, the larger the training sample size for one class is, the smaller its corresponding classification error rate is, while the smaller the sample size, the larger the classification error rate. The main causes lie in that the penalty of misclassification for each training sample is considered equally. Weighted support vector machines for classification are proposed in this paper where penalty of misclassification for each training sample is different. By setting the equal penalty for the training samples belonging to same class, and setting the ratio of penalties for different classes to the inverse ratio of the training class sizes, the obtained weighted support vector machines compensate for the undesirable effects caused by the uneven training class size, and the classification accuracy for the class with small training size is improved. Experimental simulations on breast cancer diagnosis show the effectiveness of the proposed methods.