A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> Object-oriented unit tests consist of sequences of method invocations. Behavior of an invocation depends on the state of the receiver object and method arguments at the beginning of the invocation. Existing tools for automatic generation of object-oriented test suites, such as Jtest and J Crasher for Java, typically ignore this state and thus generate redundant tests that exercise the same method behavior, which increases the testing time without increasing the ability to detect faults. This work proposes Rostra, a framework for detecting redundant unit tests, and presents five fully automatic techniques within this framework. We use Rostra to assess and minimize test suites generated by test-generation tools. We also present how Rostra can be added to these tools to avoid generation of redundant tests. We have implemented the five Rostra techniques and evaluated them on 11 subjects taken from a variety of sources. The experimental results show that Jtest and JCrasher generate a high percentage of redundant tests and that Rostra can remove these redundant tests without decreasing the quality of test suites. <s> BIB001 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> Aspect-oriented software development is gaining popularity with the adoption of languages such as AspectJ. Testing is an important part in any software development, including aspect-oriented development. To automate generation of unit tests for AspectJ programs, we can apply the existing tools that automate generation of unit tests for Java programs. However, these tools can generate a large number of test inputs, and manually inspecting the behavior of the software on all these inputs is time consuming. We propose Raspect, a framework for detecting redundant unit tests for AspectJ programs. We introduce three levels of units in AspectJ programs: advised methods, advice, and intertype methods. We show how to detect at each level redundant test that does not exercise new behavior. Our approach selects only non-redundant tests from the automatically generated test suites, thus allowing the developer to spend less time in inspecting this reduced set of tests. We have implemented Raspect and applied it on 12 subjects taken from a variety of sources; our experience shows that Raspect can effectively reduce the size of generated test suites for inspecting AspectJ programs <s> BIB002 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> There are several problem areas that must be addressed when applying randomization to unit testing. As yet no general, fully automated solution that works for all units has been proposed. We therefore have developed RUTE-J, a Java package intended to help programmers do randomized unit testing in Java. In this paper, we describe RUTE-J and illustrate how it supports the development of per-unit solutions for the problems of randomized unit testing. We report on an experiment in which we applied RUTE-J to the standard Java TreeMap class, measuring the efficiency and effectiveness of the technique. We also illustrate the use of randomized testing in experimentation, by adapting RUTE-J so that it generates randomized minimal covering test suites, and measuring the effectiveness of the test suites generated. <s> BIB003 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> Software engineering comprehends several disciplines devoted to prevent and remedy malfunctions and to warrant adequate behaviour. Testing, the subject of this paper, is a widespread validation approach in industry, but it is still largely ad hoc, expensive, and unpredictably effective. Indeed, software testing is a broad term encompassing a variety of activities along the development cycle and beyond, aimed at different goals. Hence, software testing research faces a collection of challenges. A consistent roadmap of the most relevant challenges to be addressed is here proposed. In it, the starting point is constituted by some important past achievements, while the destination consists of four identified goals to which research ultimately tends, but which remain as unreachable as dreams. The routes from the achievements to the dreams are paved by the outstanding research challenges, which are discussed in the paper along with interesting ongoing work. <s> BIB004 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> R<scp>ANDOOP</scp> for Java generates unit tests for Java code using feedback-directed random test generation. Below we describe R<scp>ANDOOP</scp>'s input, output, and test generation algorithm. We also give an overview of RANDOOP's annotation-based interface for specifying configuration parameters that affect R<scp>ANDOOP</scp>'s behavior and output. <s> BIB005 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> Combinatorial testing consists in generating (possibly large) test suites by combining both the sequencing of several operations, and the selection of test data. The TOBIAS tool is based on this generation technique. The combinatorial part of the approach makes both its strength and its weakness. Indeed, the more tests are produced, the more confidence the user may have in his/her program. Nevertheless, simple patterns may result in millions of test cases due to the combinatorial explosion problem, leading to intractable testsuites. To overcome this weakness, TOBIAS makes it possible to connect the generator to "selectors" that choose a subset of the test suite based on various techniques or criteria. This paper presents such a test suite reduction technique, based on a stochastic approach. The selection is improved so a uniform repartition of test cases is performed w.r.t. a given criterion, namely, an abstraction of the test cases based on their "shape". The approach is implemented into TOBIAS as a generic selector, independent of the system under test. It has been put into practice on a smart card case study, for which the results are reported in this paper. <s> BIB006 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> The continuous use of the Web for daily operations by businesses, consumers, and the government has created a great demand for reliable Web applications. One promising approach to testing the functionality of Web applications leverages the user-session data collected by Web servers. User-session-based testing automatically generates test cases based on real user profiles. The key contribution of this paper is the application of concept analysis for clustering user sessions and a set of heuristics for test case selection. Existing incremental concept analysis algorithms are exploited to avoid collecting and maintaining large user-session data sets and to thus provide scalability. We have completely automated the process from user session collection and test suite reduction through test case replay. Our incremental test suite update algorithm, coupled with our experimental study, indicates that concept analysis provides a promising means for incrementally updating reduced test suites in response to newly captured user sessions with little loss in fault detection capability and program coverage. <s> BIB007 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> Reducing test suites is crucial for reducing the test cost, for the test cost is directly proportional to the size of test suites. Reducing the test cost is particularly important when we need to urgently develop a stable compiler for a newly developed processor. In this paper, we propose a framework to reduce test suites for retargeted compilers using intermediate representations of test input programs. We describe that redundancy of test cases at the level of source codes exists from the viewpoint of intermediate representation and, thus, testing efficiency can be improved by excluding those redundant test cases. We also conduct a case study with RTL where we found that test suite can be considerably reduced. In addition, we show that the fault detection capability can be maintained with the reduced test suites by means of mutation testing. <s> BIB008 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> Test case management may make the testing process more efficient and thus accelerate software delivery. With the popularity of using JUnit for testing Java software, researchers have paid attention to techniques to manage JUnit test cases in regression testing of Java software. Typically, most existing test case management tools are based on the coverage information. However, coverage information may need extra efforts to obtain. In this paper, we present an Eclipse IDE plug-in (named Jtop) for managing JUnit test cases in absence of coverage information. Jtop statically analyzes the program under test and its corresponding JUnit test cases to perform the following management tasks: regression test case selection, test suite reduction and test case prioritization. Furthermore, Jtop also enables the programmer to manually manipulate test cases through a graphical user interface. <s> BIB009 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> Test-suite minimization techniques aim to eliminate redundant test cases from a test-suite based on some criteria, such as coverage or fault-detection capability. Most existing test-suite minimization techniques have two main limitations: they perform minimization based on a single criterion and produce suboptimal solutions. In this paper, we propose a test-suite minimization framework that overcomes these limitations by allowing testers to (1) easily encode a wide spectrum of test-suite minimization problems, (2) handle problems that involve any number of criteria, and (3) compute optimal solutions by leveraging modern integer linear programming solvers. We implemented our framework in a tool, called MINTS, that is freely-available and can be interfaced with a number of different state-of-the-art solvers. Our empirical evaluation shows that MINTS can be used to instantiate a number of different test-suite minimization problems and efficiently find an optimal solution for such problems using different solvers. <s> BIB010 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> Test-suite reduction technique aims to find a subset of the test suite while still satisfying the original test requirements; therefore, it can save the cost of software testing. Because many test cases have been removed, the testing information is also lost. Fault localization is a technique using testing information to locate the fault and widely used by programmers to debug programs, so it suffers from the side effects of test-suite reduction. How to reduce the test-suite size in software testing with the premise of retaining or improving fault-localization effectiveness has become the hot spot in the area of software debugging recently. In this paper, we propose a new approach that selectively keeps the limited redundant test cases in the reduced set; it makes the new reduced set relatively redundant compared to the original one, and we expect that it retains or even improves fault-localization effectiveness. We also describe a framework that implements our approach and conduct a set of empirical studies for evaluation. The results show that our approach can retain or even improve fault-localization effectiveness as expected. <s> BIB011 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> Abstract: With widespread use of embedded processors, there is an increasing need to develop compilers for them in a timely manner. Retargeting has been an effective approach to constructing new compilers by modifying the back-end of existing compilers. An efficient testing method for retargeted compilers using intermediate codes is proposed in this paper. The concepts of the intermediate-code-based approach are described, and it is demonstrated that the proposed approach is efficient and effective enough. We have developed an automated tool, PLOOSE, which generates test suites based on source-code coverage criteria and then reduces them based on intermediate-code coverage. In addition, case studies are presented which reveal that a test suite based on source codes can be considerably reduced using intermediate codes. Moreover, by mutation analysis, it was found that the size of the test suite has been reduced by over 90% on average, but the fault detection capability of the reduced test suites is still approximately 80% of that of the original test suites. In particular, the proposed reduction method can be useful for testing compilers in the early stage of the development of retargeted compilers. <s> BIB012 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> A program fails. What now? Taking a single failing run, we record and minimize the interaction between objects to the set of calls relevant for the failure. The result is a minimal unit test that faithfully reproduces the failure at will: "Out of these 14,628 calls, only 2 are required". In a study of 17 real-life bugs, our JINSI prototype reduced the search space to 13.7% of the dynamic slice or 0.22% of the source code, with only 1--12 calls left to examine. <s> BIB013 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> When software developers make changes to a program, it is possible that they will introduce faults into previously working parts of the code. As software grows, a regression test suite is run to ensure that the old functionality still works as expected. Yet, as the number of test cases increases, it becomes more expensive to execute the test suite. Reduction and prioritization techniques enable developers to manage large and unwieldy test suites. However, practitioners and researchers do not always use and study these methods due, in part, to a lack of availability. In response to this issue, this paper describes an already released open-source framework that supports both research and practice in regression testing. The sharing of this framework will enable the replication of empirical studies in regression testing and encourage faster industrial adoption of these useful, yet rarely used, techniques. <s> BIB014 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> Post-deployment in situ testing and validation techniques have become an important means of ensuring the reliability of mobile and embedded systems. However, these techniques do not take into consideration the amount of energy they consume, which is an issue of paramount concern for systems with limited energy budgets. In this paper we propose a new test suite minimization approach that allows developers to generate energy-efficient, minimized test suites. The approach is based on encoding minimization problems as integer linear programming problems. Our empirical evaluation shows that, compared to traditionally generated minimized test suites, the test suites generated by our approach are equally effective in terms of their test coverage, but can realize energy savings of up to 90 %. <s> BIB015 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> Limitations imposed by the traditional practice in financial institutions of running risk analysis on the desktop mean many rely on models which assume a normal Gaussian distribution of events which can seriously underestimate the real risk. In this paper, we propose an alternative service which uses the elastic capacities of Cloud Computing to escape the limitations of the desktop and produce accurate results more rapidly.The Business Intelligence as a Service (BIaaS) in the Cloud has a dual-service approach to compute risk and pricing for financial analysis. The first type of BIaaS service uses three APIs to simulate the Heston Model to compute the risks and asset prices, and computes the volatility (unsystematic risks) and the implied volatility (systematic risks) which can be tracked down at any time. The second type of BIaaS service uses two APIs to provide business analytics for stock market analysis, and compute results in the visualised format, so that stake holders without prior knowledge can understand. A full case study with two sets of experiments is presented to support the validity and originality of BIaaS. Additional three examples are used to support accuracy of the predicted stock index movement as a result of the use of the Heston Model and its associated APIs.We describe the architecture of deployment, together with examples and results which show how our approach improves risk and investment analysis and maintaining accuracy and efficiency whilst improving performance over desktops. <s> BIB016 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> This paper examines how cloud computing can be used in the area of brain segmentation with regard to satisfactory technical and user evaluations. It explains eleven APIs associated with each brain segment and the process of capturing data for each segment. Functionality and experiments associated with each API are discussed. Dancing is to capture data more easily. Results are used to explain why some segments are more active in dancing, with two evaluations undertaken. The first evaluation is the use of brain segmentations developed for medical cloud computing education (MCCE) and results confirm that cloud computing offers a 20% improvement in learning satisfaction. The second evaluation is focused on recapturing a lost skill. Results confirm that volunteers have their heartbeat, blood pressure, emotion, body coordination and vision at their peak. Benefits of using cloud brain segmentation technology are presented to illustrate positive impacts to healthcare informatics, education and cost reduction. <s> BIB017 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> When comparing Cloud and non-Cloud Storage it can be difficult to ensure that the comparison is fair. In this paper we examine the process of setting up such a comparison and the metric used. Performance comparisons on Cloud and non-Cloud systems, deployed for biomedical scientists, have been conducted to identify improvements of efficiency and performance. Prior to the experiments, network latency, file size and job failures were identified as factors which degrade performance and experiments were conducted to understand their impacts. Organizational Sustainability Modeling (OSM) is used before, during and after the experiments to ensure fair comparisons are achieved. OSM defines the actual and expected execution time, risk control rates and is used to understand key outputs related to both Cloud and non-Cloud experiments. Forty experiments on both Cloud and non-Cloud systems were undertaken with two case studies. The first case study was focused on transferring and backing up 10,000 files of 1 GB each and the second case study was focused on transferring and backing up 1000 files 10 GB each. Results showed that first, the actual and expected execution time on the Cloud was lower than on the non-Cloud system. Second, there was more than 99% consistency between the actual and expected execution time on the Cloud while no comparable consistency was found on the non-Cloud system. Third, the improvement in efficiency was higher on the Cloud than the non-Cloud. OSM is the metric used to analyze the collected data and provided synthesis and insights to the data analysis and visualization of the two case studies. Organizational sustainability modeling (OSM) compares Cloud and non-Cloud storage.We identify factors affect performance and design ways to make fair comparisons.We explain how to use OSM including its definitions, input and output.We present two case studies of Big Data storage with 40 runs to support.Results are analyzed and presented with data analysis and visualization. <s> BIB018 </s> A survey on Test Suite Reduction frameworks and tools <s> Problem: Find a Reduced Suite (RS) containing minimal test cases from TS that satisfies all test requirements Ri at least once. <s> The SocialMedia API has been developed to demonstrate proofs-of-concept.Cybernetics-1 and cybernetics-2 functions are developed for cybernetics Social Cloud.Large scale simulations have been tested for capacity tests. No costs are involved.All the steps involved are fully justified for research contributions. This paper proposes a Social Cloud, which presents the system design, development and analysis. The technology is based on the BOINC open source software, our hybrid Cloud, Facebook Graph API and our development in a new Facebook API, SocialMedia. The creation of SocialMedia API with its four functions can ensure a smooth delivery of Big Data processing in the Social Cloud, with four selected examples provided. The proposed solution is focused on processing the contacts who click like or comment on the author's posts. Outputs result in visualization with their core syntax being demonstrated. Four functions in the SocialMedia API have evaluation test and each client-server API processing can be completed efficiently and effectively within 1.36źs. We demonstrate large scale simulations involved with 50,000 simulations and all the execution time can be completed within 70,000źs. Cybernetics functions are created to ensure that 100% job completion rate for Big Data processing. Results support our case for Big Data processing on Social Cloud with no costs involved. All the steps involved have closely followed system design, implementation, experiments and validation for Cybernetics to ensure a high quality of outputs and services at all times. This offers a unique contribution for Cybernetics to meet Big Data research challenges. <s> BIB019
The optimal TSR problem is known to be NP-complete problem and is equivalent to set cover problem  . Researchers have recommended complete automation support for various activities of test suite development cycle, such as test generation, execution, and evaluation, to minimize high cost of regression testing. Bertolino BIB004 has recommended 100% automated testing, which facilitates the tester to meet the challenges of a resource constrained testing environment. Motivated by this, researchers have proposed various frameworks that results various tools to provide automation facility during TSR process (Horgan and London, 1992 BIB001 BIB002 BIB003 BIB005 , Jaygarl et al., 2010 BIB009 BIB006 BIB014 BIB007  BIB008 BIB012 BIB010 BIB015 BIB011 BIB013 . Consequently, automated TSR helps in accelerating the software delivery process compared to manual test filtering BIB010 ). Automation has been achieved for Cloud storage for big data, which provide a supporting use case BIB018 . Cloud service APIs Plays a vital role to integrate enterprise applications as they offer a set of protocols, which helps in connecting applications to various cloud services. APIs are useful for different disciplines such as business intelligence BIB016 and social networks BIB019 BIB017 . Prior works Harman, 2012, Elberzhager et al., 2012) lack in considering and analyzing TSR frameworks that is necessary to understand the body of knowledge in the area of TSR. To the best of our knowledge, this is the first effort that studies the TSR frameworks and tools comprehensively. The main objective of this survey is to provide an up-to-date view and in-depth analysis of state-of-the-art that is necessary to understand the body of knowledge. The survey analyzes, synthesizes, and categorizes current state-of-the-art TSR frameworks and tools. Furthermore, the survey focuses on identifying future research opportunities in this field of study. The main contributions of the paper are listed below:  An extensive review on automated support for TSR.  Presenting a thematic taxonomy to categorize the existing literature based on various testing domains, approaches, and their corresponding parameters.  Providing a detailed comparative analysis of TSR frameworks based on the devised taxonometric parameters.  Highlighting the strengths and limitations of the TSR tools and frameworks related to a particular testing domain.  Synthesizing current state-of-the-art based on the underlying common philosophies.  Highlighting several potential research issues in this field of study. The rest of the paper is organized as follows: Section 2 presents a thematic taxonomy of TSR frameworks. Section 3 discusses current state-of-the-art TSR frameworks/tools based on various testing domains and approaches. Furthermore, it discusses strengths and weaknesses of existing TSR frameworks. Section 4 provides a comparison of current TSR frameworks based on the devised thematic taxonomy. Section 5 discusses potential research issues followed by concluding remarks in Section 6.