Software Testing Process Models Benefits&Drawbacks: a Systematic Literature Review <s> TM12: <s> Many organizations and industries continuously struggle to achieve a foundation of a sound verification and validation process. Because verification and validation is only marginally addressed in software process improvement models like CMMI, a separate verification and validation improvement model has been developed. The framework of the improvement model presented in this paper is based on the Testing Maturity Model (TMM). However considerable enhancements are provided, such as a V&V assessment procedure and a detailed metrics program to determine effectiveness and efficiency of the improvements. This paper addresses the development approach of MB-V2M2, an outline of the model, early validation experiments, current status and future outlook. The resulting model is dubbed MB-V2M2 (Metrics Based Verification@Validation Maturity Model). <s> BIB001 </s> Software Testing Process Models Benefits&Drawbacks: a Systematic Literature Review <s> TM12: <s> The objective of this qualitative study was to understand the complex practice of software testing, and based on this knowledge, to develop process improvement propositions that could concurrently reduce development and testing costs and improve software quality. First, a survey of testing practices was onducted and 26 organizational units (OUs) were interviewed. From this sample, five OUs were further selected for an in-depth case study. The study used grounded theory as its research method and the data was collected from 41 theme-based interviews. The analysis yielded improvement propositions that included enhanced testability of software components, efficient communication and interaction between development and testing, early involvement of testing, and risk-based testing. The connective and central improvement proposition was that testing ought to adapt to the business orientation of the OU. Other propositions were integrated around this central proposition. The results of this study can be used in improving development and testing processes. <s> BIB002 </s> Software Testing Process Models Benefits&Drawbacks: a Systematic Literature Review <s> TM12: <s> Existing test process assessment and improvement models intend to raise maturity of an organization with reference to testing activities. Such process assessments are based on ldquowhatrdquo testing activities are being carried out, and thus implicitly evaluate process quality. Other test process measurement techniques attempt to directly assess some partial quality attribute such as efficiency or effectiveness some test metrics. There is a need for a formalized method of test process quality evaluation that addresses both implicitly and partially of these current evaluations. This paper describes a conceptual framework to specify and explicitly evaluate test process quality aspects. The framework enables provision of evaluation results in the form of objective assessments, and problem-area identification to improve the software testing processes. <s> BIB003 </s> Software Testing Process Models Benefits&Drawbacks: a Systematic Literature Review <s> TM12: <s> It is well known that most of mission critical systems like defense systems demand higher quality than general software systems. In order to assure the high quality of the systems, those software systems must be thoroughly tested with well defined testing processes. As software systems evolve, software testing should also be improved with continual-base. Even if there are some previous researches on test process model, those models have some limitations and problems to be applied for software testing process improvement in a specific domain like military weapon systems. In this paper, we propose a new testing maturity model, called MND-TMM (Ministry Of National Defense-Testing Maturity Model), which can be adopted effectively for the weapon software system development. We also present OWL ontology for the MND-TMM, called MTO(MND-TMM ontology) to help software organizations set up their effective testing strategies and improve their testing processes, which can lead to high quality product developments. <s> BIB004 </s> Software Testing Process Models Benefits&Drawbacks: a Systematic Literature Review <s> TM12: <s> As one of the most challenging and costly processes, the software testing provides strong support for the high quality software. The popular process improvement models do not adequately address testing application issues, nor has the knowledge management (KM) implementation in testing been well defined. The plan-do-check-action (PDCA)-based testing improvement framework is proposed to address these issues. The framework contains the adaptive processes definition, the processes implementation and measurement analysis, and benchmarking-based assessment approach. Experiments show that: the framework can stimulate team member's subjective initiative, promote the software testing process and improve the testing service quality. <s> BIB005 </s> Software Testing Process Models Benefits&Drawbacks: a Systematic Literature Review <s> TM12: <s> One of the latest additions in defining the test process is the upcoming ISO/IEC 29119 standard, which aims to define a universally applicable generic test process model. However, currently the standard does not offer any support for the adoption process of the model. In this paper, we present our framework, which aims to combine a maturity level-based approach with the standard process. Our objective was to create an easy-to-use framework for organizations to assess how closely their existing test process follows the standard, and give feedback on improvement objectives. Our results indicate that combining maturity levels with the standard is a viable approach to assess the implementation of the standard in practice. <s> BIB006 </s> Software Testing Process Models Benefits&Drawbacks: a Systematic Literature Review <s> TM12: <s> The discipline of software testing is increasingly drawing industry's attention to itself due to the demand for quality products and the high level of competition in the market. This scenario can be supported by a maturity model that continuously improves the discipline of testing when applied to the software development lifecycle. Therefore, MPT.BR, the acronym in Portuguese for Improving Test Processing. Brazil, is presented as a maturity model for testing to help when introducing testing disciplines in software organizations. It can be seen as an alternative for companies that aim to increase their product quality and ROI. MPT.BR has been applied over the last 3 years in small and medium-sized software organizations, the results of which will be discussed in this paper. <s> BIB007 </s> Software Testing Process Models Benefits&Drawbacks: a Systematic Literature Review <s> TM12: <s> Automated test generation is gaining popularity in the software industry, largely due to its labor-saving benefits and its ability to achieve high test coverage. The introduction of this technology into an organization does not, however, always meet with success. One reason for this is often the fact that the testing process and organization are not adjusted accordingly. Thus, in order for an organization to successfully pursue automated test generation, the test process must also be improved to enable this development. In this paper, we introduce an automated test generation add-on for the popular test process improvement model, TPI. We also present a baseline TPI profile for successful introduction of automated test generation. <s> BIB008
The knowledge management strategy needs to be included at the organizational level BIB005 . TM15: The model needs further development and studies for validity BIB006 specifying evaluation scope and measurements. Then, procedures for the evaluation activities are defined and as the last step the evaluation is executed and measurements are collected BIB003 . Jacobs and Trienekens BIB001 discuss generic practices of the MB-VV-MM model [TM13] and group them into several common features. The first group is Commitment to Perform, which includes a practice for defining organizational expectations for the process. The second feature Ability to Perform ensures availability of needed resources and assigning the responsibility for performing the process and achieving results. Furthermore, training is performed to ensure the necessary skills to execute the process. The feature Activities Performed describes the activities that have to be executed to establish the process. Configuration and process measurements are managed within the Directing Implementation feature. The process is monitored and controlled to collect information and if necessary, perform corrective actions. The last feature Verifying Implementation evaluates if the process is implemented as it was planned and provides business management BIB001 . Small Organizations. The practices of MPT.BR [TM6] for small organizations are discussed in the study of Furtado et al. BIB007 . It includes generic practices for the first two maturity levels (Partially managed and Managed). Other three levels do not add extra generic practices. A generic practice reflects process capabilities that are important for process areas of maturity levels BIB007 . Another model designed for small organizations is MTPF [TM17], which practices are described in the study of Karlström et al.  . The model defines several categories and related phases. At the beginning test planning is performed with the aim to create a test plan, coordinate software quality assurance, involve risk management and Automated Testing. The Automated Testing domain is specific for the ATG add-on for TPI model [TM16] and it is discussed in the research of Heiskanen et al. BIB008 . The practices are defined separately for all maturity levels and key areas. We collect and map them into our phases. During the planning phase, the project is initialized and a plan is created. Then, it is important to design and model test basis and subsequently to conduct techniques and environment. After performing tests, the strategy is evaluated and optimal test automation is conducted. The monitoring and control phase includes conducting organization metrics and risk and defect management BIB008 . Military Systems. The MND-TMM model [TM7] is designed and developed for military systems. The study of Ryu et al. BIB004 provides practices only for the first phase. The goal of the planning phase is to set a test plan which describes the activities needed at a specific level, and identify software development life cycle performed at each level BIB004 . Automation or Telecommunication Domains. The Observing Practice [TM10] was applied in the Automation and telecommunication domain. Its practices are presented in the reseach of Taipale and Smolander BIB002 . It is a very specific approach, which is based on collecting data through interviews. From that reason, the practices represent steps performed during the interviews. The first step includes selecting case organizational units, e.g. the study focuses on the units that develop and test technical software for automation or telecommunication domains in Finland. As a next step, data are collected by conducting interviews with respondents to elicit views and experiences. Subsequently, data are analyzed by using the grounded theory and based on the results process improvements are proposed and applied BIB002 . Third party testing center. The third party testing center domain is presented in the PDCA-based software testing improvement framework [TM12] . The study of Xu-Xiang and Wen-Ning BIB005 describes the framework process in several phases. The process starts with building the learning organization and requirements analysis. The process-oriented plan is prepared which includes adaptive testing processes based on the organizational processes. Then, it is possible to implement the plan and collect data, which are analyzed with the aim to identify process improvements. When improvement actions are prepared they can be implemented and the results are assessed BIB005 . Automotive software. The study of Kasoju et al.  discusses the practices of the Evidence-based Software Engineering approach [TM14] specific for automotive software. The authors define test process activities based on the analysis of collected data. The planning phase includes estimation of the requirements, test techniques, tools and other test artifacts, and creation of a test plan. During the analysis and design phase, test scripts and test data are identified and designed to determine how the tests will be performed. Then, required test environment and other important artifacts have to be collected and built. When everything is prepared, tests can be executed and reports of results can be send to the customer  .