Interaction information
The interaction information is a generalization of the mutual information for more than two variables. There are many names for interaction information, including amount of information, information correlation, co-information, and simply mutual information. Interaction information expresses the amount of information (redundancy or synergy) bound up in a set of variables, beyond that which is present in any subset of those variables. Unlike the mutual information, the interaction information can be either positive or negative. These functions, their negativity and minima have a direct interpretation in algebraic topology. Definition The conditional mutual information can be used to inductively define the interaction information for any finite number of variables as follows: where Some authors define the interaction information differently, by swapping the two terms being subtracted in the preceding equation. This has the effect of reversing the sign for an odd number of variables. For three variables , the interaction information is given by where is the mutual information between variables and , and is the conditional mutual information between variables and given . The interaction information is symmetric, so it does not matter which variable is conditioned on. This is easy to see when the interaction information is written in terms