An extended systematic literature review on provision of evidence for safety certification <s> RELATED WORK <s> Multidomain software-intensive system certification is a growing concern in international standards organizations. Certification is traditionally based on domain-specific technical and operational requirements, historical information about the reliability of the different subsystems, and historical accident data, but these may not be available for interdomain systems. There is a need for international and interdisciplinary legal recognition of certification authorities. The author explores both the legal and technical aspects of this critical, timely problem. <s> BIB001 </s> An extended systematic literature review on provision of evidence for safety certification <s> RELATED WORK <s> Search-based software testing is the application of metaheuristic search techniques to generate software tests. The test adequacy criterion is transformed into a fitness function and a set of solutions in the search space are evaluated with respect to the fitness function using a metaheuristic search technique. The application of metaheuristic search techniques for testing is promising due to the fact that exhaustive testing is infeasible considering the size and complexity of software under test. Search-based software testing has been applied across the spectrum of test case design methods; this includes white-box (structural), black-box (functional) and grey-box (combination of structural and functional) testing. In addition, metaheuristic search techniques have also been applied to test non-functional properties. The overall objective of undertaking this systematic review is to examine existing work into non-functional search-based software testing (NFSBST). We are interested in types of non-functional testing targeted using metaheuristic search techniques, different fitness functions used in different types of search-based non-functional testing and challenges in the application of these techniques. The systematic review is based on a comprehensive set of 35 articles obtained after a multi-stage selection process and have been published in the time span 1996-2007. The results of the review show that metaheuristic search techniques have been applied for non-functional testing of execution time, quality of service, security, usability and safety. A variety of metaheuristic search techniques are found to be applicable for non-functional testing including simulated annealing, tabu search, genetic algorithms, ant colony methods, grammatical evolution, genetic programming (and its variants including linear genetic programming) and swarm intelligence methods. The review reports on different fitness functions used to guide the search for each of the categories of execution time, safety, usability, quality of service and security; along with a discussion of possible challenges in the application of metaheuristic search techniques. <s> BIB002 </s> An extended systematic literature review on provision of evidence for safety certification <s> RELATED WORK <s> Increasingly, licensing and safety regulatory bodies require the suppliers of software-intensive, safety-critical systems to provide an explicit software safety case – a structured set of arguments based on objective evidence to demonstrate that the software elements of a system are acceptably safe. Existing research on safety cases has mainly focused on how to build the arguments in a safety case based on available evidence; but little has been done to precisely characterize what this evidence should be. As a result, system suppliers are left with practically no guidance on what evidence to collect during software development. This has led to the suppliers having to recover the relevant evidence after the fact – an extremely costly and sometimes impractical task. Although standards such as the IEC 61508 – which is widely viewed as the best available generic standard for managing functional safety in software – provide some guidance for the collection of relevant safety and certification information, this guidance is mostly textual, not expressed in a precise and structured form, and is not easy to specialize to context-specific needs. To address these issues, we present a conceptual model to characterize the evidence for arguing about software safety. Our model captures both the information requirements for demonstrating compliance with IEC 61508 and the traceability links necessary to create a seamless chain of evidence. We further describe how our generic model can be specialized according to the needs of a particular context, and discuss some important ways in which our model can facilitate software certification. <s> BIB003
Several papers discuss the notion of evidence in specific situations and how this evidence can be structured and assessed. We do not treat these as related work but rather as the primary studies for our SLR. The discussions in this section are therefore targeted at contrasting our work with the more generic classifications of safety evidence as well as the relevant existing SLRs. Some threads in previous work, e.g. BIB003 , address the problem of safety evidence classification through focusing on safety standards such as IEC61508. Further threads, e.g.  , consider the structuring of evidence for safety cases. A safety case is a structured argument aimed at providing a compelling, comprehensive, and valid case that a system is safe for a given application in a given operating environment  . The arguments in a safety case are always accompanied by evidence supporting the arguments. More recently, there has been an OMG initiative called SACM aimed at standardizing the notion of and the concepts related to assurance evidence and arguments  . While the above threads have been a useful start for the current SLR, they are either too specific (relating to only one standard or application domain) or do not provide a thorough and sufficiently detailed analysis of the possible evidence types and how to structure and assess them. There are a number of SLRs in the literature whose scope partially overlaps with ours, e.g., on testing BIB002 , on requirements specification  , and on reliability  . None of these specifically address the topic of evidence for safety. Some past work attempts to compare safety standards in different domains with the aim of identifying the commonalities and differences among them  [3] BIB001 . However, these comparisons are limited in scope and, in contrast to ours, are not based on a systematic review. In summary, little has been done to date by way of synthesising and summarising, in a comprehensive manner, the state of the art on safety evidence. Consequently, no unifying framework exists for reasoning about and communicating safety evidence. This observation led us to the need for the SLR as a way to gain new insights into how to specify, structure and assess safety evidence.