Mutation analysis can effectively capture the dependency between source code and test results. This has been exploited by Mutation Based Fault Localisation (MBFL) techniques. However, MBFL techniques suffer from the need to expend the high cost of mutation analysis after the observation of failures, which may present a challenge for its practical adoption. We introduce SIMFL (Statistical Inference for Mutation-based Fault Localisation), an MBFL technique that allows users to perform the mutation analysis in advance against an earlier version of the system. SIMFL uses mutants as artificial faults and aims to learn the failure patterns among test cases against different locations of mutations. Once a failure is observed, SIMFL requires either almost no or very small additional cost for analysis, depending on the used inference model. An empirical evaluation of SIMFL using 355 faults in Defects4J shows that SIMFL can successfully localise up to 103 faults at the top, and 152 faults within the top five, on par with state-of-the-art alternatives. The cost of mutation analysis can be further reduced by mutation sampling: SIMFL retains over 80 of its localisation accuracy at the top rank when using only 10 of generated mutants, compared to results obtained without sampling.
As a fault localisation technique, can be seen as predicting where faults are most likely to be located. This is different from, but related to, other forms of software defect prediction @cite_40 @cite_41 which typically uses software metrics and code attributes, but rarely information from the dynamic testing, to predict defect-proneness. A recent exception is the study by @cite_7 , which uses test results to dynamically tune traditional defect prediction models. Still, they predict and rank modules based on their overall defect proneness, with the goal of directing quality assurance efforts, rather than for localising actually observed faults.
@cite_41 @cite_40 @cite_7
2151666086 2105776892 2803652181
Background: The accurate prediction of where faults are likely to occur in code can help direct test effort, reduce costs, and improve the quality of software. Objective: We investigate how the context of models, the independent variables used, and the modeling techniques applied influence the performance of fault prediction models. Method: We used a systematic literature review to identify 208 fault prediction studies published from January 2000 to December 2010. We synthesize the quantitative and qualitative results of 36 studies which report sufficient contextual and methodological information according to the criteria we develop and apply. Results: The models that perform well tend to be based on simple modeling techniques such as Naive Bayes or Logistic Regression. Combinations of independent variables have been used by models that perform well. Feature selection has been applied to these combinations when models are performing particularly well. Conclusion: The methodology used to build models seems to be influential to predictive performance. Although there are a set of fault prediction studies in which confidence is possible, more studies are needed that use a reliable methodology and which report their context, methodology, and performance comprehensively. Software defect prediction strives to improve software quality and testing efficiency by constructing predictive classification models from code attributes to enable a timely identification of fault-prone modules. Several classification models have been evaluated for this task. However, due to inconsistent findings regarding the superiority of one classifier over another and the usefulness of metric-based classification in general, more research is needed to improve convergence across studies and further advance confidence in experimental results. We consider three potential sources for bias: comparing classifiers over one or a small number of proprietary data sets, relying on accuracy indicators that are conceptually inappropriate for software defect prediction and cross-study comparisons, and, finally, limited use of statistical testing procedures to secure empirical findings. To remedy these problems, a framework for comparative software defect prediction experiments is proposed and applied in a large-scale empirical comparison of 22 classifiers over 10 public domain data sets from the NASA Metrics Data repository. Overall, an appealing degree of predictive accuracy is observed, which supports the view that metric-based classification is useful. However, our results indicate that the importance of the particular classification algorithm may be less than previously assumed since no significant performance differences could be detected among the top 17 classifiers. Abstract Test resource constraints is a common phenomenon in software testing. Using defect prediction to guide the resource allocation can significantly improve the efficiency and effectiveness of available test resources. However, traditional defect prediction (t-DP) is a static strategy, where the predictor cannot be dynamically adjusted during the software testing process (STP). This paper combines defect prediction with feedback control in STP and proposes a feedback-based defect prediction model, where the test results generated during STP is used as feedback information for on-line adjustment of predictor to optimize the prediction result. In addition, a novel approach called feedback-based integrated prediction (FIP) is proposed to improve the prediction accuracy, where a global predictor and a local predictor are employed to make an integrated prediction using the weight to adjust the effects of predictors at different test stages. A systematic experiment is conducted to investigate the performance of the FIP over 10 public data sets. Results show that FIP has better prediction efficiency and better robustness for external data than the t-DP, especially when the percentage of the test modules is 40 .