An extended systematic literature review on provision of evidence for safety certification <s> Checklists (16%): <s> This paper deals with the approach for assessing the safety critical computer-based railway systems. It presents the method used for planning and conducting an assessment. This paper provides a precise and workable assessment criteria of safety critical digital architecture. It lists the generic measures/techniques used to construct a safe architecture. The objective of this paper is to present the activities related to the assessment of safety digital architectures. It gives a set of criteria used for judging the compliance with the quality and the safety requirements. The criteria are drawn from standards requirements and best practices. These criteria have been applied to assess various case studies used in the European project named ACRuDA (Assessment and Certification Rules for Digital Architecture). <s> BIB001 </s> An extended systematic literature review on provision of evidence for safety certification <s> Checklists (16%): <s> Standards for safety critical software usually either mandate or recommend development and assessment techniques which are deemed appropriate to reduce the risk of flaws in the software contributing to accidents. These recommendations are usually broken down into a number of "levels" of rigour, with the highest levels being applied where the consequences of failure, or risk, are most severe. The paper discusses the extent to which it is possible to find evidence that there is a genuine variation in risk with level, i.e. that the principles in the standards are sound, and questions some of the assumptions underlying these standards.The paper then goes on to discuss the potential advantages of using product-based evidence to demonstrate safety of software, as opposed to relying on process prescription. It outlines current work on developing and applying "evidence frameworks" as alternatives to the process-based approach, and identifies some of the challenges in gaining widespread acceptance of such approaches.Finally the paper discusses the ALARP principle, and what would be necessary to show that risks associated with safety-critical software have been reduced ALARP. The paper concludes that there are some fundamental difficulties with applying the ALARP principle to software, which neither the process nor evidence-based approaches to demonstrating software safety can readily resolve. <s> BIB002 </s> An extended systematic literature review on provision of evidence for safety certification <s> Checklists (16%): <s> When software safety requirements are imposed on legacy safety-critical systems, retrospective safety cases need to be formulated as part of recertifying the systems for further use and risks must be documented and managed to give confidence for reusing the systems. The SEI software development risk taxonomy focuses on general software development issues. It does not, however, cover all the safety risks. The software safety risk taxonomy was developed which provides a construct for eliciting and categorizing software safety risks in a straightforward manner. In this paper, we present extended work on the taxonomy for safety that incorporates the additional issues inherent in the development and maintenance of safety-critical systems with software. An instrument called a software safety risk taxonomy based questionnaire (TBQ) is generated containing questions addressing each safety attribute in the software safety risk taxonomy. Software safety risks are surfaced using the new TBQ and then analyzed. In this paper we give the definitions for the specialized product engineering class within the software safety risk taxonomy. At the end of the paper, we present the tool known as the 'legacy systems risk database tool' that is used to collect and analyze the data required to show traceability to a particular safety standard. <s> BIB003 </s> An extended systematic literature review on provision of evidence for safety certification <s> Checklists (16%): <s> Formal methods are powerful specification and verification techniques for establishing high confidence in safety-critical systems. However, there are a number of concerns about the use of evidence generated from formal methods, when used in place of conventional testing, for satisfying certain certification objectives. In this paper, we address this issue by reviewing two certification documents, DO-178B and the UK Defence Standard 00-56, focusing on their approach to accepting formal analytical evidence. We also present a generic goal-based safety case that can be instantiated to facilitate the justification and presentation of formal analysis to the certification authorities. The safety case is based on claims about (1) the achievement of the intents of the certification objectives, (2) the demonstration of the trustworthiness of formal analysis and (3) the practical feasibility of deploying formal methods within a specific project. <s> BIB004 </s> An extended systematic literature review on provision of evidence for safety certification <s> Checklists (16%): <s> Formal methods have emerged as a complementary approach to ensuring quality and correctness of high-confidence medical systems, overcoming limitations of traditional validation techniques such as simulation and testing. In this paper, we propose a new methodology to obtain certification assurance for complex medical systems design, based on the use of formal methods. The methodology consists of five main phases: first, informal requirements, resulting in a structured version of the requirements, where each fragment is classified according to a fixed taxonomy. In the second phase, informal requirements are represented in formal modeling language, with a precise semantics, and enriched with invariants and temporal constraints. The third phase consists of refinement-based formal verification to test the internal consistency and correctness of the specifications. The fourth phase is the process of determining the degree to which a formal model is an accurate representation of the real world from the perspective of the intended uses of the model using model-checker. Last phase provides an animation framework for the formal model with real-time data set instead of toy-data, and offers a simple way for specifiers to build a domain specific visualization that can be used by domain experts to check whether a formal specification corresponds to their expectations. Furthermore, we show the effectiveness of this methodology for modeling of a cardiac pacemaker system. <s> BIB005 </s> An extended systematic literature review on provision of evidence for safety certification <s> Checklists (16%): <s> The assurance, reuse, and recertification of legacy safety-critical computer systems are problems affecting government and industry. The assurance and recertification processes involve gathering existing data from such systems, and evaluating how the data aids in meeting the intent of software safety requirements imposed on the systems after the fact. A Software Safety Risk Taxonomy and Software Safety Risk Evaluation (SSRE) process was used in four projects at three NASA Centers for the identification, analysis, consolidation and mitigation of software safety risks associated with meeting only a subset of the requirements of the NASA-STD-8719.13B Software Safety Standard. A Legacy Systems Risk Database was built to manage the projects data and create the traceability between a safety process improvement model (+SAFE), the NASA software safety requirements, the safety taxonomy and software safety risks. This paper describes the steps to perform the SSRE, the initial design of the database showing how the requirements traceability is maintained and some select research study results. <s> BIB006 </s> An extended systematic literature review on provision of evidence for safety certification <s> Checklists (16%): <s> Many safety-critical aircraft functions are software-enabled. Airborne software must be audited and approved by the aerospace certification authorities prior to deployment. The auditing process is time-consuming, and its outcome is unpredictable, due to the criticality and complex nature of airborne software. To ensure that the engineering of airborne software is systematically regulated and is auditable, certification authorities mandate compliance with safety standards that detail industrial best practice. This paper reviews existing practices in software safety certification. It also explores how software safety audits are performed in the civil aerospace domain. The paper then proposes a statistical method for supporting software safety audits by collecting and analysing data about the software throughout its lifecycle. This method is then empirically evaluated through an industrial case study based on data collected from 9 aerospace projects covering 58 software releases. The results of this case study show that our proposed method can help the certification authorities and the software and safety engineers to gain confidence in the certification readiness of airborne software and predict the likely outcome of the audits. The results also highlight some confidentiality issues concerning the management and retention of sensitive data generated from safety-critical projects. <s> BIB007
We classify in this category techniques that introduce a "to-do list" consisting of a set of guided questions that need to be answered or checked while reviewing the evidence. The questions could, for example, be a set of conditions that must be met in order to gain confidence in the evidence collected and to check its sufficiency BIB004 . We identified different variations of checklists in the literature: Design Checklists BIB005 , which assess evidence based on the design of the system. GQM-based checklists BIB007 , which are based on the Goal/Question/Metric measurement framework  . They define top-level goals for assessing product and process evidence, questions to be answered to achieve these goal and metrics providing a measurable reference against which analysis can be performed. Argumentation-based checklists BIB002 , which assess evidence by mixing checklists with argumentation. The Taxonomy-Based Questionnaire BIB006 , which contains 305 questions addressing the safety attributes and artefacts in the Software Safety Risk Taxonomy and Software Safety Risk Evaluation process BIB003 . Plain Checklists BIB001 , which are checklists that do not fall under any of the more specific variations discussed above.