Bug prediction is the process of training a machine learning model on software metrics and fault information to predict bugs in software entities. While feature selection is an important step in building a robust prediction model, there is insufficient evidence about its impact on predicting the number of bugs in software systems. We study the impact of both correlation-based feature selection (CFS) filter methods and wrapper feature selection methods on five widely-used prediction models and demonstrate how these models perform with or without feature selection to predict the number of bugs in five different open source Java software systems. Our results show that wrappers outperform the CFS filter; they improve prediction accuracy by up to 33 while eliminating more than half of the features. We also observe that though the same feature selection method chooses different feature subsets in different projects, this subset always contains a mix of source code and change metrics.
Shivaji al @cite_0 study the impact of multiple feature selection techniques (filters and wrappers) on the performance of Na "ive Bayes and Support Vector Machines when classifying code changes as buggy or clean. They report a significant enhancement in the accuracy measures of both classifiers when feature selection is applied. However, Shivaji al consider as a feature @cite_0 . This includes variable names, method names, keywords, comment words, They end up with a staggering number of features ranging from 6,127 to 41,942 features. The vast majority of features in the initial feature set are bound to be irrelevant, hence the results of their study are exaggerated and cannot be generalized.
@cite_0
2050496630
Machine learning classifiers have recently emerged as a way to predict the introduction of bugs in changes made to source code files. The classifier is first trained on software history, and then used to predict if an impending change causes a bug. Drawbacks of existing classifier-based bug prediction techniques are insufficient performance for practical use and slow prediction times due to a large number of machine learned features. This paper investigates multiple feature selection techniques that are generally applicable to classification-based bug prediction methods. The techniques discard less important features until optimal classification performance is reached. The total number of features used for training is substantially reduced, often to less than 10 percent of the original. The performance of Naive Bayes and Support Vector Machine (SVM) classifiers when using this technique is characterized on 11 software projects. Naive Bayes using feature selection provides significant improvement in buggy F-measure (21 percent improvement) over prior change classification bug prediction results (by the second and fourth authors [28]). The SVM's improvement in buggy F-measure is 9 percent. Interestingly, an analysis of performance for varying numbers of features shows that strong performance is achieved at even 1 percent of the original number of features.