A survey on Test Suite Reduction frameworks and tools <s> Automated Fault-Localization based TSR Tools <s> Where the creation, understanding, and assessment of software testing and regression testing techniques are concerned, controlled experimentation is an indispensable research methodology. Obtaining the infrastructure necessary to support such experimentation, however, is difficult and expensive. As a result, progress in experimentation with testing techniques has been slow, and empirical data on the costs and effectiveness of techniques remains relatively scarce. To help address this problem, we have been designing and constructing infrastructure to support controlled experimentation with testing and regression testing techniques. This paper reports on the challenges faced by researchers experimenting with testing techniques, including those that inform the design of our infrastructure. The paper then describes the infrastructure that we are creating in response to these challenges, and that we are now making available to other researchers, and discusses the impact that this infrastructure has and can be expected to have. <s> BIB001 </s> A survey on Test Suite Reduction frameworks and tools <s> Automated Fault-Localization based TSR Tools <s> Test-suite reduction technique aims to find a subset of the test suite while still satisfying the original test requirements; therefore, it can save the cost of software testing. Because many test cases have been removed, the testing information is also lost. Fault localization is a technique using testing information to locate the fault and widely used by programmers to debug programs, so it suffers from the side effects of test-suite reduction. How to reduce the test-suite size in software testing with the premise of retaining or improving fault-localization effectiveness has become the hot spot in the area of software debugging recently. In this paper, we propose a new approach that selectively keeps the limited redundant test cases in the reduced set; it makes the new reduced set relatively redundant compared to the original one, and we expect that it retains or even improves fault-localization effectiveness. We also describe a framework that implements our approach and conduct a set of empirical studies for evaluation. The results show that our approach can retain or even improve fault-localization effectiveness as expected. <s> BIB002 </s> A survey on Test Suite Reduction frameworks and tools <s> Automated Fault-Localization based TSR Tools <s> A program fails. What now? Taking a single failing run, we record and minimize the interaction between objects to the set of calls relevant for the failure. The result is a minimal unit test that faithfully reproduces the failure at will: "Out of these 14,628 calls, only 2 are required". In a study of 17 real-life bugs, our JINSI prototype reduced the search space to 13.7% of the dynamic slice or 0.22% of the source code, with only 1--12 calls left to examine. <s> BIB003 </s> A survey on Test Suite Reduction frameworks and tools <s> Automated Fault-Localization based TSR Tools <s> Software testing occurs simultaneously during the software development to detect errors as early as possible and to guarantee that changes made in software did not affect the system negatively. However, during the development phase, the test suite is updated and tends to increase in size. Due to the resource and time constraints for re-executing large test suites, it is important to develop techniques to reduce the effort of regression testing. Several approaches have been studied to reduce the effort of regression testing: test suite minimization, selection, and prioritization. Test suite minimization techniques aims at identifying and eliminating redundant test cases from the suite. Test suite selection techniques identifies a subset of test cases from suite, required to re-test the changes in the software. Test suite prioritization techniques schedule test cases for execution in an order to increase the early fault detection. Although, the approaches present the literature have some limitations (resources, times, etc.). In this manuscript we propose a new approach to the minimization, selection and prioritization problem that try to overcomes these limitation. Our approach allows for (1) easy encoding of the relation between a test case and a component (source code statement) in a coverage matrix, (2) mapping this matrix in a set of constraints, and (3) computing optimal solutions (with the same coverage and fault detection of original set of tests) to minimization and selection problems by leveraging a fast and scalable constraint solver. We implemented our approach in a Eclipse plug-in called GZOLTAR, which is perfectly integrated in the IDE and provides functions to debugging a software program using visualization in OpenGL. Features for regression tests were implemented with the name Regression-Zoltar, RZOLTAR for short. To test our toolset, we conducted experiments to measure the efficiency and the size of regression suite. In particular, we evaluated our tool using real-world and large programs, as well as a user study in order to measure its usability. In our empirical evaluation, we show how RZOLTAR can be used to instantiate large-programs and efficiently find an solution for such problems. Our experiments show that our approach can significantly reduce the size of original test suite, and a corresponding reduction in the cost of regression testing. We also make a user study to find if we are in right way in terms of usability and functionalities. <s> BIB004
To the best of our knowledge, researchers have proposed three tools including SrTC BIB002 , JINSI BIB003 and GZoltar , to improve fault-localization effectiveness with the distribution uniformity of test cases.  SrTC: Zhang et al. BIB002 presented very first time the concept of relative redundancy for TSR and focused to balance the uneven distribution in the RS. The authors proposed a framework supported by SrTC tool, which intentionally retained a small number of redundant test cases to improve fault-localization effectiveness. SrTC contains three modules: (i) Reduction Processor, (ii) w Processor, and (iii) Evaluation, which uses and produces five types of data: (1) T, (2) REP, (3) TIE, (4) CAN, and (5) REL-REP. To judge the efficacy of SrTC, the authors conducted an experimental study by using NanoXM BIB001 as a subject program. According to the reported results, the average value of reduction was 24.25%. However, the fault-localization effectiveness was improved significantly.  JINSI: Burger and Zeller BIB003 proposed a JINSI tool that uses passing and failing execution information in order to determine faulty program entities. First of all, JINSI records and replays interactions of a given objects set and effectively reproduces the failure inducing method calls. After that, JINSI filters the sequence of failure inducing method calls by combining event slicing and delta debugging techniques, which result in a minimized object interaction. Eventually, the minimal set of unit tests is generated using the reduced interaction information. The authors experimentally evaluated the performance of the JINSI tool by applying it on six different Java subject programs. Finally, they reported that JINSI achieved an average of 0.22% search space reduction of the original Java program and produced a test driver containing eight to twelve failurereproducing interactions.  GZoltar: Campos et al. presented a GZoltar toolset that is a spectrum-based fault localization plug-in for the Eclipse Integrated Development Environment (IDE). The proposed tool supports a constraint-based approach BIB004 to perform TSR, while ensuring the same code coverage compared to the original test suite. Moreover, GZoltar facilitate the user to prioritize the RS based on the test execution time and cardinality. It also automatically produces different visual representations of the diagnostic report, which ultimately facilitate the users in detecting the most suspicious parts of the program's source code. As a result, the fault localization process can be improved using minimal possible time. GZoltar provides the infrastructure to instruments the source code of the SUT in order to automatically generate runtime data. Next, the test cases are executed to collect the code coverage data and execution traces. Finally, the collected coverage information is passed to constraint solver in order to determine minimal RS.