Context: Software performance is a critical non-functional requirement, appearing in many fields such as mission critical applications, financial, and real time systems. In this work we focused on early detection of performance bugs, our software under study was a real time system used in the mobile advertisement marketing domain. Goal: Find a simple and easy to implement solution, predicting performance bugs. Method: We built several models using four machine learning methods, commonly used for defect prediction: C4.5 Decision Trees, Naive Bayes, Bayesian Networks, and Logistic Regression. Results: Our empirical results show that a C4.5 model, using lines of code changed, file's age and size as explanatory variables, can be used to predict performance bugs (recall = 0.73, accuracy = 0.85, and precision = 0.96). We show that reducing the number of changes delivered on a commit, can decrease the chance of performance bug injection. Conclusions: We believe that our approach can help practitioners to eliminate performance bugs early in the development cycle. Our results are also of interest to theoreticians, establishing a link between functional bugs and (non-functional) performance bugs, and explicitly showing that attributes used for prediction of functional bugs can be used for prediction of performance bugs.
The work that is closest to ours is by @cite_18 , which studied a set of 109 real-world performance bugs. They studied the bugs’ lifetime from inception to fix, their root causes and introduction mechanisms, in order to create rule-based detectors. We used a comparable approach to create detectors (referred to as "patterns" in our study) with the aim to identify similar performance bugs. They studied software written in Java, C, C++, and JavaScript; ours was written in Python Programming language does affect performance of the models, as shown by @cite_15 . . Moreover, we focused on finding an efficient method to automatically detect performance bugs based on data extracted through the use of patterns (197 real-world performance bugs in our case). Summarizing, @cite_18 focused on analyzing characteristics of performance bugs, while in this study, our focus is on understanding the contribution of each source code attribute, as extracted from the source code repository, to the predicting power of the several machine learning algorithms that we used. Lastly, our general approach and the methodology for creating the prediction model were meant to be easily reproducible in the future. Therefore, our work is complementary.
@cite_18 @cite_15
2145458045 2021509116
Developers frequently use inefficient code sequences that could be fixed by simple patches. These inefficient code sequences can cause significant performance degradation and resource waste, referred to as performance bugs. Meager increases in single threaded performance in the multi-core era and increasing emphasis on energy efficiency call for more effort in tackling performance bugs. This paper conducts a comprehensive study of 110 real-world performance bugs that are randomly sampled from five representative software suites (Apache, Chrome, GCC, Mozilla, and MySQL). The findings of this study provide guidance for future work to avoid, expose, detect, and fix performance bugs. Guided by our characteristics study, efficiency rules are extracted from 25 patches and are used to detect performance bugs. 332 previously unknown performance problems are found in the latest versions of MySQL, Apache, and Mozilla applications, including 219 performance problems found by applying rules across applications. We compare the effectiveness of four modeling methods--negative binomial regression, recursive partitioning, random forests and Bayesian additive regression trees--for predicting the files likely to contain the most faults for 28 to 35 releases of three large industrial software systems. Predictor variables included lines of code, file age, faults in the previous release, changes in the previous two releases, and programming language. To compare the effectiveness of the different models, we use two metrics--the percent of faults contained in the top 20 of files identified by the model, and a new, more general metric, the fault-percentile-average. The negative binomial regression and random forests models performed significantly better than recursive partitioning and Bayesian additive regression trees, as assessed by either of the metrics. For each of the three systems, the negative binomial and random forests models identified 20 of the files in each release that contained an average of 76 to 94 of the faults.