Software Product Lines (SPLs) are inherently difficult to test due to the combinatorial explosion of the number of products to consider. To reduce the number of products to test, sampling techniques such as combinatorial interaction testing have been proposed. They usually start from a feature model and apply a coverage criterion (e.g. pairwise feature interaction or dissimilarity) to generate tractable, fault-finding, lists of configurations to be tested. Prioritization can also be used to sort generate such lists, optimizing coverage criteria or weights assigned to features. However, current sampling prioritization techniques barely take product behaviour into account. We explore how ideas of statistical testing, based on a usage model (a Markov chain), can be used to extract configurations of interest according to the likelihood of their executions. These executions are gathered in featured transition systems, compact representation of SPL behaviour. We discuss possible scenarios and give a prioritization procedure validated on a web-based learning management software.
Statistical testing of software establishes a basis for statistical inference about a software system's expected field quality. This paper describes a method for statistical testing based on a Markov chain model of software usage. The significance of the Markov chain is twofold. First, it allows test input sequences to be generated from multiple probability distributions, making it more general than many existing techniques. Analytical results associated with Markov chains facilitate informative analysis of the sequences before they are generated, indicating how the test is likely to unfold. Second, the test input sequences generated from the chain and applied to the software are themselves a stochastic model and are used to create a second Markov chain to encapsulate the history of the test, including any observed failure information. The influence of the failures is assessed through analytical computations on this chain. We also derive a stopping criterion for the testing process based on a comparison of the sequence generating properties of the two chains. > Markov chains with Labelled Transitions can be used to generate test cases in a model-based approach. These test cases are generated by random walks on the model according to probabilities associated with transitions. When these probabilities correspond to a usage profile, reliability may be estimated. However, in early stages of development, such probabilities are not easy to determine, thus default profiles must be considered. In such a case it may be interesting to target some coverage criteria rather to use classical uniform probability generation approach. In this paper we enrich an existing industrial tool based on usage profile with 3 possibilities to create default profiles that improve transition coverage. We report experiments that compare the improvement of the coverage rates by our approaches with respect to uniform probabilities on transitions from a given state, which is the current default profile. Software validation embodies two notions: fault removal and fault forecasting. Statistical testing involves exercising a piece of software by supplying it with input values that are randomly selected according to a defined probability distribution on its input domain. It can be used as a practical tool for revealing faults in a fault removal phase, and for assessing software dependability in a fault forecasting phase. In both of these, its efficiency is linked to the adequacy of the input probability distribution with respect to the test experiment goal. In this paper a mixed validation strategy combining deterministic and random test data is defined, and the theoretical and experimental work performed to support the strategy is reported. The quoted results relate to the unit testing of four real programs from the nuclear field. They confirm the high fault revealing power of statistical structural testing. Two main directions for further investigation of statistical testing are indicated by the reported work. These are described and solutions to the associated problems are outlined. Operational profiles are an important part of the technology and practice of software reliability engineering. The concept was developed originally ( 1987) to make it possible to specify the nature of the use of a software-based system so that testing could be made as realistic as possible and so that reliability measurements would reflect that realism. However, the operational profile rapidly became useful for additional purposes in software reliability engineering (Musa 1993). In fact, it is also proving useful for purposes outside of software reliability engineering as well. This paper gives an overview of operational profile practice, discussing what the operational profile is, why it is important, and how it is developed and applied. It also presents some current open research questions; work in these areas can be expected to affect the practice of the future. We propose a novel way of automating statistical structural testing of software, based on the combination of uniform generation of combinatorial structures, and of randomized constraint solving techniques. More precisely, we show how to draw test cases which balance the coverage of program structures according to structural testing criteria. The control flow graph is formalized as a combinatorial structure specification. This provides a way of uniformly drawing execution paths which have suitable properties. Once a path has been drawn, the predicate characterizing those inputs which lead to its execution is solved using a constraint solving library. The constraint solver is enriched with powerful heuristics in order to deal with resolution failures and random choice strategies.