Assurance and Evaluation Method for Reliability Requirements Completion of Aviation Product Based on CMM

Modern aviation product needs to meet the reliability requirements during the development process. In this paper, a comprehensive assurance system of reliability requirements completion is established and a method to evaluate the assurance system capabilities of reliability requirements completion based on CMM (Capability Maturity Model) is proposed. The evaluate index system in each level is described and AHP method is applied to determine the weight of each index factor in maturity. The evaluation results can comprehensively and effectively direct the aviation product reliability work during the development process and ultimately ensure the reliability of aviation product meets the requirements.


Introduction
As the complexity of modern aviation product increases, reliability work needs to be implemented. The goal of reliability work is to ensure that the aviation product meets the specified reliability requirements and maintains and improves its reliability level. Further to ensure the system can meet the requirements of operational readiness and mission success and the requirements for guaranteed resources can reduce [1]. The reliability requirements refer to a series of reliability qualitative and quantitative requirements and specific reliability work items requirements required by the ordering party in the contract. Capability maturity model related reliability have been partially studied. Researchers at the University of Maryland have used CMM to evaluate the reliability capabilities of electronics manufacturers [2][3][4]. The concept of reliability-centered system engineering capability was putting forward and the weapon equipment reliability-centered system engineering capability maturity model framework was established in the paper of researcher Kang Rui [5]. Li Guosheng added the maintainability and supportability to the reliability engineering capability maturity model [6]. This paper studies the factors that ensure the reliability requirements completed and builds the capability maturity model and proposes the evaluation method.

Assurance system of reliability requirements completion
The assurance system of reliability requirements completion refers to a series of engineering organization activities implemented in the development process which to ensure the completion of reliability requirements of aviation product. From the perspectives of organization, technology, and process, the hierarchy model of the assurance system of reliability requirements completion is shown as figure 1. The overall assurance work of reliability requirements completion of aviation product is divided into two categories which are the work of the implementation process and the work of the management process. The work of the management process is to serve the work of the implementation process. In order to establish the assurance system, the reliability requirements of product and the reliability work specified in national military standards are analyzed firstly. Then the factors affecting the completion of reliability requirements are summarized. Finally, the correlation among the factors is considered and the structure of the assurance system is determined. With further practice and deeper learning, the assurance system will be further refined.

Assurance capability of reliability requirements completion evaluation
This paper defines the assurance capability of reliability requirements completion as the ability to develop the aviation product to meet the specified reliability requirements under the constraints on various factors of the assurance system of reliability requirements completion. A which means assurance capability maturity of reliability requirements completion is used to measure the system capability and A is a number within [0, 5]. In this paper, fuzzy comprehensive evaluation method and analytic hierarchy process (AHP) method are used to evaluate assurance capability maturity of reliability requirements completion. The evaluation system construction idea is shown in figure 2.

Maturity level
Maturity level is the core content of the capability maturity mode [7]. According to the principle of traditional capability maturity models and referring to the CMMI continuous representation model, ISO/IEC 15504 and other models [8][9], the hierarchical framework is established as shown in figure 3. Maturity level classification and level description: Reliability assurance activities are basically in a chaotic state with no goals.
Reliability assurance activities are controlled by execution. Focusing on projectlevel specifications.
Reliability assurance activities have been organized and institutionalized. The basic process of activities has been tailored from the standard system of the unit.
Reliability assurance activities can be quantified managed.
Data collection, evaluation and testing can be executed. (1) Initial level. Reliability assurance activities are executed under the chaotic management state or not executed. The organization has the reliability assurance activities without corresponding norms and mature goals. Work is not performed in accordance with the principles and ideas of reliability assurance. Activities such as complete planning, tracking, analysis, improvement and experience dissemination are not implemented. Work steps are not systematic. Reliability assurance activities and the outputs obtained are partially valid. Many problems exist, including unidentified information on the collaborative process across units, geographies, and interdisciplinary, invalid of complex product development and management, and uncertainty project success.
(2) Management level. The typical project is the core at most of the time and project-level specifications are focused. The reliability assurance activity program is developed specifically including objectives, plans, and processes. The requirements for reliability assurance activities are clearly defined and activities are planned, tracked, evaluated and accepted. Reliability assurance organizations are established and organized with relevant guidelines to support and encourage. Activity resources are allocated. Appropriate corrective actions exist when there is a deviation from the operational effect and the plan or the process is not executed. The successful experience of similar projects in the past can be reproduced.
(3) Definition level. The standards-suitable reliability assurance activity process is fully established. Project-level activities are absorbed, summarized and improved. The processes related to reliability assurance activities are clearly defined at the organizational level. The project reliability assurance activities are obtained through reductions of organizational standard activities and reasonable adjustment according to the actual technical status of the project. The tailored activity contributes activity improvement information to the organization's active asset library. The biggest feature of this level is standardization. (4) Quantization level. Reliability assurance activities are measurable and quantifiable. Statistics and other quantitative methods are used as management means and quantitative quality, process and performance indicators are used as the standard for management. Existing defects can be identified  4 and trends in work processes can be predicted. Measures can be implemented before problems arise to reduce costs. Good software tools are used as support generally. (5) Optimization level. Reliability Assurance activities are able to meet current and anticipated goals and accommodate changes. Innovative methods and techniques are used centrally. It is possible to achieve continuous improvement activities by actively collecting data and screening and analyzing the data. Activity defects can be identified and defect patterns can be analyzed. The root cause of defects can be discovered and corresponding improvements can be taken. The defect prevention plan was developed and tracked. Reliability assurance staff are continuously trained and upgraded. The activity process is the result of continuous optimization. The quantification level and the optimization level are based on the first three levels and the processes are further quantitatively predicted and controlled and analyzed and optimized according to the cost, efficiency and quality requirements of the organization's activities. Quantization level and optimization level are relatively difficult to achieve, and they are stages to realize potential by actively discovering and solving problems in an open and positive environment.

Evaluation index
The evaluation indexes are established based on the factors of the assurance system of reliability requirements completion. The evaluation indexes are shown in table 1.

Weight calculation of evaluation indexes
This paper uses the AHP method to establish a weight set, which classifies various related factors in a complex problem into an interrelated and orderly hierarchy. Based on certain objective judgments, the first-level, the second-level and the third-level assessment indexes are compared and analyzed. In the weight calculation of the three layers of assessment indexes, some experts are invited to construct the discriminant matrix by comparing the index among each other. 1~9 are used as comparison scales among evaluation factors. By solving these discriminant matrices, it can be calculated that the three weights of the first-level assessment index which are denoted as ( =1,2,3). The weights of the second-level assessment index for the first-level assessment indexes are denoted as ( =1,2,3; =1,2⋯). The weights of the third-level assessment index for the second-level assessment indexes are denoted as k ( =1,2,3; =1,2⋯; k=1,2⋯). The weight set of index is denoted as W ={ 1 , 2 , …, k }( =1,2,3; =1,2⋯).

Scoring criteria of index
In this paper, fuzzy comprehensive evaluation method is used to determine the scoring criteria shown as table 2.

Maturity calculating
(1) A membership function whose span is [0,1] to quantify index and experts scores are used to get fuzzy evaluating matrix with the third-level assessment index. Degree of membership indicates the membership degree of the maturity level of factor . (1) where d is the number of experts, is the number of experts whose score is . (2) The fuzzy evaluating matrix is shown as equation (2): (3) Maturity evaluation vector of the second-level assessment index is shown as equation (3): (3) (4) Maturity evaluation vector of the first-level assessment index is shown as equation (4): (4) (5) System evaluation value is shown as equation (5):

Result analysis
The system evaluation value A can reflect the level of assurance capability maturity of reliability requirements completion. The corresponding description can be found in figure 3.

Conclusion
This paper aims to help the aviation product meet the reliability requirements. An assurance system of reliability requirements completion and a method to evaluate the assurance system capabilities of reliability requirements completion based on CMM of aviation product are proposed. The result can not only reflect the level of assurance system of reliability requirements completion, but also expanded the application of capability maturity model in reliability. Further study will enrich the assurance system of reliability requirements completion.