Entropy Applications to Water Monitoring Network Design: A Review <s> Total Correlation <s> A multivariate analysis based on transmitted information is presented. It is shown that sample transmitted information provides a simple method for measuring and testing association in multi-dimensional contingency tables. Relations with analysis of variance are pointed out, and statistical tests are described. <s> BIB001 </s> Entropy Applications to Water Monitoring Network Design: A Review <s> Total Correlation <s> A set λ of stochastic variables, y1, y2,..., yn, is grouped into subsets, µ1, µ2,..., µk. The correlation existing in λ with respect to the µ's is adequately expressed by C= Σi=1k S(µi)-S(λ)≥0, where S(v) is the entropy function defined with reference to the variables y in subset v. For a given λ, C becomes maximum when each µi consists of only one variable, (n=k). The value Cis then called fhe total correlation in λ, Ctot(λ). The present paper gives various theorems, according to which Ctot(λ) can be decomposed in terms of the partial correlations existing in subsets of λ, and of quantities derivable therefrom. The information-theoretical meaning of each decomposition is carefully explained. As illustrations, two problems are discussed at the end of the paper: (1) redundancy in geometrical figures in pattern recognition, and (2) randomization effect of shuffling cards marked "zero' or "one." <s> BIB002
While transinformation and mutual information have the same definition, total correlation is not equivalent to them as the total correlation is a simple estimate that defines the amount of shared information typically of multiple variables. Simply, the total correlation is defined by the difference between the sum of marginal entropy of N variables and their joint entropy BIB001 BIB002 , which is given as: If N = 2 in Equation (14), the total correlation will be equal to the transinformation or mutual information. However, the transinformation is only meaningful to two random variables as shown in Equations (11) to (13); therefore, the total correlation and the transinformation values would be different if N > 2.