Method of multi-source testing information fusion based on bayesian networks

of weapon systems and the conversion of inertial navigation test information, which provides model reference and possible solutions to the Bayesian network based multi-source information fusion methods. Bayesian networks are a very general and powerful tool that can be used for a large number of problems involving uncertainty: reasoning, learning, planning and perception. They provide a language that supports efficient algorithms for the automatic construction of expert systems in several different contexts. The range of applications of Bayesian networks currently extends over almost all fields including engineering, biology and medicine, information and communication technologies and finance. This book is a collection of original contributions to the methodology and applications of Bayesian networks. It contains recent developments in the field and illustrates, on a sample of applications, the power of Bayesian networks in dealing the modeling of complex systems. Readers that are not familiar with this tool, but have some technical background, will find in this book all necessary theoretical and practical information on how to use and implement Bayesian networks in their own work. There is no doubt that this book constitutes a valuable resource for engineers, researchers, students and all those who are interested in discovering and experiencing the potential of this major tool of the century.


Introduction
Large Equipment System Acceptance (such as reliability and accuracy, etc.) is a complicated system engineering. In order to check the performance of large equipment, multi-perspective and more-approach tests are adopted to get a variety of test information. These information are related to many aspects, such as the information in different phases of design, development, pilot, production and application phase, test information of products in different levels (systems, subsystems, components) and the history information of test-related products. These different but still interrelated information have brought a great more reference to analysis and assessment of large-scale equipment, and meanwhile those uncertain information would brought more risk to the assessment of decision-making. How to integrate these information of multiple sources effectively to make an objective evaluation on the performance of large-scale equipments has been a great challenge to the engineering researchers. For example, in assessing the reliability of weapons systems, the cost of system-level testing is often too much which limits the number of test times. In that condition the test information of various equipment and subsystems are urged to be fully utilized. Similarly, in order to improve the practical accuracy of INS (inertial navigation system), a series of checking from the test phase to application phase such as ground calibration tests, vehicle-loaded tests, aircraft-loaded tests and missile-loaded tests, are requisite and the outcome should function to the error coefficients estimation. In the circumstance of sufficient test data, the classical approach of comprehensive assessment to reliability has been widely used; while in contrast when test data are insufficient and moreover they present multi-stage and multi-level properties, the classical approach is in effectiveness challenge. With the development of computer technology and improvement of Bayes methods, especially the emergence of MCMC (Markov chain Monte Carlo) methods and WinBUGS (Bayesian inference Using Gibbs Sampling) software, the Bayesian network is more and more popularized in the application of multi-source information fusion [1~3]. The Bayesian network is a causal network, which is used as an inference engine for the calculation of beliefs or probability of events given the observations of other events in the same network. It does not only make good use of model information and sample data, but also integrates the unknown parameters in the overall distribution of information. Besides, it has overcome defects of traditional static model being incapable of handling emergencies. These flexible, easy to adapt to external changes features can make up for the shortage of insufficient poor quality samples brought to traditional statistical methods, so it is more suitable for prediction and reality reveal to models. The most attractive feature of the Bayesian network is given an observation for one node, the statistical information for all nodes would be updated. This feature is very valuable in the context of model validation, when experimental observations may not be available on the final model output but may be available on one or more intermediate quantities. This paper presents a new approach of information fusion used Bayesian network and is organised as follows. The background of this research, especially for the application in reliability assessment and precision evaluation, is introduced in section 1. In section 2, the fundamental of Bayesian network is stated and how to establish networks for a typical case are then illustrated. In section 3, it is emphasized in utilizing Bayesian networks to integrate multi-source testing information obtained from different layers, states and environments, where the examples of reliability parameters estimation for weapon system and information conversion for inertial navigation system error model are simulated to show the effectiveness of the scheme presented. Finally, some conclusions are given in the end.

Bayesian inference
The basic idea of Bayesian inference is to express the uncertainty of all the unknown parameters of the model by probability distributions [4]. This means that an unknown parameter is modeled as a random parameter beforehand. are in the text the random parameters of our interest is denoted as The observed variables, or the evidence Actually the joint distribution model described in equation (6) consists of network of conditional dependencies between random variables. Such networks are often called Bayesian networks. A Bayesian network can be represented as a directed acyclic graph, in which elliptic nodes correspond to random variables and rectangular nodes represent constants and directed arcs between the nodes describe the dependence between the parameters. Moreover, a solid arrow indicates a stochastic dependence while a hollow arrow indicates a logical function. As an example the graphical representation of the hierarchical model described by equations (3)~(6) is depicted as a Bayesian network in Figure 1.

Implementation of the proposed method
In Bayesian models, where we are interested in the relationships of a large number of variables, Bayesian network becomes an appropriate representation. A Bayesian network is a graphical model that efficiently encodes the joint probability distribution for a large set of variables. However, determining the conditional posterior distributions for the parameters of interest is usually not a simple task in Bayesian networks. To obtain an analytic result for the conditional posterior distribution the denominator of the Bayes formula, which normalizes the conditional posterior distribution to unity, must be evaluated. A proportional result for the posterior distribution can be obtained without resolving the denominator, but the integral for the numerator is only one dimension less. For analytic result, or at least for a good approximation of the result, the integrals have to be determined in a way or another. For simple models the integrals can be evaluated using conventional numerical techniques, but in most applications the Bayesian network contain tens and hundreds of parameters and the analytic evaluation of the integrals by conventional numerical techniques is impossible. Therefore, an MCMC [5] approach is used for obtaining the posterior distribution. In MCMC methods, Monte Carlo estimates of probability density functions and expected values of the desired quantities are obtained using samples generated by a Markov chain whose limiting distribution is the distribution of interest. Thus one can generate samples of multiple random variables from a complicated joint probability density function without explicitly evaluating or inverting the joint cumulative density function. Several schemes such as Metropolis-Hastings algorithm, Gibbs sampling, etc. are available to carry out MCMC simulations [6]. Gibbs sampling is commonly utilized due to its simplicity in the implementation. Let x denote a vector of k random variables x . To sample quantities from the full conditional density of the ith variable, the following relationship is used: Gibbs sampling can then be used to sequentially generate samples from the joint probability density function using the full conditional densities, as below: Step 1: Initialize Step2: Generate Step3: 1 j j   ; Step4: End if j reaches the maximum number of runs, or else, return to step 2.
Gibbs sampling has been shown to have geometric convergence of order N (number of runs) [4]. Exact full conditional densities may not always be available. In such cases, the Gibbs sampling procedure is supplementary to the Metropolis-Hastings algorithm. During each run, the full conditional density function ( )

Testing information fusion using Bayesian networks
Since Bayesian networks can easily establish the uncertainty relationships among parameters and update all the prior distributions of the random variables once new observations come out, it is a effective solution to multi-source information fusion. In this section, two representative applications of Bayesian networks to weapon system reliability evaluation and INS testing information conversion under different circumstance are discussed as illustration. Note that the modeling and simulations in this paper are carried out using the WinBUGS program, and so all the Bayesian networks presented below are depicted in the WinBUGS format. For closer review about the WinBUGS program, see Spiegelhalter et al. [7].

Reliability evaluation of weapon system
Since a great deal of manpower and material resources are requisite in system-level tests to reliability evaluation for such complex weapon system, whereas much more convenience would be obtained if in unit-level test case, the engineering practice usually adopts reliability information of composition units to analyze the reliability of the entire system. These unit-level test information make up for the lack of information on system-level test, and reduce the number of tests in the premise of sustaining its confidence effectively. Obviously, to evaluate weapon system reliability in Bayes method is a kind of information fusion. More clearly, reliability test information about unit and system should be fused into the posterior distribution of system reliability first, and based on it the Bayesian statistical inference could then be carried out. To facilitate following discussion, suppose the weapon system is pass-fail series system.

Reliability analysis of pass-fail unit
The pass-fail unit likelihood function is In the discussion of binomial distribution, the prior distribution of reliability is often in Beta distribution, i.e.
where a and b are auxiliary parameters, Since these auxiliary parameters reflect the full utilization of prior information, selection of a and b are very critical for reliability analysis. Martz et al [8] When m is small, the sampling error may yield negative value in (11) of ( ) a b  , so that it is amended as Once the auxiliary parameters of prior distribution are determined, using Bayes' rule there would be where D is the experimental data, n is the number of tests, f indicates the number of failure. It is obviously that the posterior probability density function The Bayesian analysis method for unit reliability is deduced in above discussion. The following would proceed to system reliability calculation for pass-fail series system.

Series system synthesis
Assume all the reliability tests of system or units considered is in pass-fail type. The series system consists of p units, and denotes i  as the reliability of the constituent units, thus Denote i n as the number of unit tests, i x as the number of success, so the system reliability is Assume i  is independent of each other, and prior distribution of ( ) are the distribution parameters. The posterior probability density function of the system's reliability  is induced as If there are few units in series system, the above formula for the series' system reliability evaluation is feasible; while if the cell number is in great many, the calculations could not be sustainable any more. The system encountered in engineering practice used to be composed in many units, and the information obtained are comprised of unit reliability test information and system reliability information, in this condition the abovementioned method is incapable in handling complex tests information. The following would introduce the reliability analysis method for this kind of complex system using Bayesian network. Assume the distribution of system reliability is also subject to Beta, thus the posterior joint probability density function of unit's reliability i  and system reliability  can be rewritten as: Since the above form of joint distribution is too complex, Bayesian network of the system reliability is established in assistance. In this Bayesian network, MCMC sampling method is employed to update the network graph, hence the analysis to posterior distribution of reliability could be implemented as soon as Markov chain is stabilized. Take the three-numbered pass-fail series as an example, the Bayesian network is developed as below.  , , , R R R R represent the system's reliability and units' reliability;    Table 1. In the sample sequence of system reliability of Figure 3, the Markov chain has been shown fused completely and furthermore reached steady state in sampling 2,000 times. Figure 4 shows the profile of posterior density distribution estimation of reliability which is depicted in consistent with Beta distribution explicitly. Bayesian network integrates test information and prior information about the units and system, and the information in each node is then disseminated to the entire network through the directed link, therefore the integrated inference about the test information is realized. The advantage of this approach is that reliability statistical analysis of the system in it would be more accurate. And furthermore, the percentile information such as the upper bound of reliability are also obtained through MCMC sampling, in result the system reliability analysis is more comprehensive and effective.

Testing information fusion of INS
When the tests are implemented under different technical conditions, the error coefficients of inertial navigation system may have different statistical characteristics. This paper presents a method of multi-source testing information fusion for inertial navigation system based on Bayesian network, which might provide a new idea to the precision evaluation work. Firstly, the testing information of all sorts is interrelated to each other by circumstance-conversion-factor, and then a graphic mapping model is constructed to represent the relationship of all variables by using Bayesian network. With testing information, the post statistical characteristics of variables such as circumstance-conversion-factor can be rapidly inferred by MCMC algorithm applied in Bayesian network, and consequently information conversion of inertial navigation system error model could be carried out between different testing conditions. As the test information are related to the temperature, pressure, humidity and other circumstance factors, a standard state for each type of test should be selected beforehand. In this condition, all the information would be conversed to be the one in the corresponding standard state first, and then conversed in the reference of standard state information.   which way the established Bayesian network for information fusion is depicted as Figure 6.

Inference of circumstance conversion factor
For the sake that the tests of INS could not be a great many, there are only 10 groups of ground calibration and 5 groups of vehicle-loaded estimates to error factor generated through the simulation, see Table 2. Note that a new data generation would accompany a set of mean and standard deviation production.
Ground calibration true distribution of variables in data production 0     Table 3, where the true distribution characteristics (mean and standard deviation) of 1  , 0  and 1  may be computed out from those of other variables.
Obviously, in the case of 15 groups of observational data, Bayesian network has effectively fused the information obtained from calibration tests and vehicle-loaded tests. In comparison with prior distribution, the characteristics of the posterior distribution of all variables whether mean or standard deviation is much closer to those in real situation. Summarized from posterior statistical properties, estimates of the mean for each variable is slightly better than that of standard deviation, while the posterior inference to i shown inferior to that of K and i  . Given the limited sample size, accomplish system-level test data fusion utilizing Bayesian network is still quite effective despite of the errors exist between posterior inference and the true data.   about error coefficient were induced from ground calibration tests (see Table 4). In this condition, we wish to infer the statistical characteristics of error coefficient represents 1   in vehicle-loaded test in the light of circumstance factor K . The links among the variables are depicted in Bayesian network of Figure 15.
Ground calibration tests data true distribution of variables in data generation 0   As long as our purpose is to do the error factor conversion, so we focus on the posterior inference to 1   , K and 0  merely. According to the using means of prior information, two different conversion methods are adopted respectively.

A. Conversion method 1
At first, deal with the posterior statistical characteristics of the variables as the prior information of current coefficient in conversion, by making use of the data fusion results prior to technology improvement merely. Then update Bayesian network as shown in Figure 15 based on the improved calibration test data. Statistical inference results are obtained and displayed in Table 5.  Table 5. Statistical results in use of prior information merely As indicated from Table 5, Circumstance factor of K is hardly changed, mean of error coefficient 0  differs slightly, so there is a big gap between the posterior statistical characteristics and the real states. Although the posterior distribution of the test data is slightly "pulled back" to the real state by the novel test data, the effect is still not very obvious. That's because compared to the prior distribution on one hand, the prior distribution is more certain (standard deviation is small), and on the other the sample information is too limited, so that prior information plays a leading role in the posterior statistical inference. The novel test information is greatly weakened by the prior, which yields inferior posterior inference of 0  . In the presence of large deviations of posterior inference, the statistical results about the conversion value of error factor is not that credible in this occasion.

B. Conversion method 2
Allowing for the impact of prior information to posterior statistical inference, especially in the occasion of small samples, the error coefficient conversion is dealt with improved prior information. These improvements include two aspects. At first, in spite the prior information of error coefficient may vary from the state of current system due to technical progress, the variation is not too much so that the mean of prior distribution could be remained. Second, by increasing the standard deviation of the prior distribution, the prior information could be "fuzzed up" so that "over-conservative" posterior inference from "over-certain" prior characteristic would be avoided; but note that the standard deviation should not be set too large, otherwise the system would tend to non-informative prior and lose the useful information. The statistical inference results from improved prior information are shown in Table 6.
In contrast to the results in Table 5, the posterior statistical inference in Table 5 Table 6. Statistical results in use of improved prior information The error coefficient conversion has been done in two methods from preceding texts analysis. Since very few samples are available, the first method takes more advantage of prior information so that the impact of test information to posterior inference is very weak; while the second method of "fuzzes up" prior information to reduce its impact on posterior inference by increasing the standard deviation, in return the impact of test information is got increased. In the premise of small but capable of accurately reflecting the true statistical characteristics samples, the above examples prove the second method is better in error coefficient conversion than the first. However, when large deviations exist between sample information and the real distribution, the risk of the using the second method increases in accompany; therefore, it is not reasonable to say that the second method is certainly better than the first.

Conclusions
MCMC technology has brought a revolutionary breakthrough to the development and application of Bayes statistical theory. Especially the emergence and further promotion of WinBUGS software, which gets the Bayesian network inference of model parameters out of complicated high-dimensional integral calculations, has routinized the analysis and application of Bayesian network. This paper has discussed the reliability assessment of weapon systems and the conversion of inertial navigation test information, which provides model reference and possible solutions to the Bayesian network based multi-source information fusion methods.