Quantum relative entropy

In quantum information theory, quantum relative entropy is a measure of distinguishability between two quantum states. It is the quantum mechanical analog of relative entropy.
Motivation.
For simplicity, it will be assumed that all objects in the article are finite-dimensional.
We first discuss the classical case. Suppose the probabilities of a finite sequence of events is given by the probability distribution "P" = {"p"1..."p""n"}, but somehow we mistakenly assumed it to be "Q" = {"q"1..."q""n"}. For instance, we can mistake an unfair coin for a fair one. According to this erroneous assumption, our uncertainty about the "j"-th event, or equivalently, the amount of information provided after observing the "j"-th event, is
The (assumed) average uncertainty of all possible events is then
On the other hand, the Shannon entropy of the probability distribution "p", defined by
is the real amount of uncertainty before observation. Therefore the difference between these two quantities
is a measure of the distinguishability of the two probability distributions "p" and "q". This is precisely the classical relative entropy, or Kullback–Leibler divergence:
Note 
Definition.
As with many other objects in quantum information theory, quantum relative entropy is defined by extending the classical definition from probability distributions to density matrices. Let "ρ" be a density matrix. The von Neumann entropy of "ρ", which is the quantum mechanical analog of the Shannon entropy, is given by
For two density matrices "ρ" and "σ", the quantum relative entropy of "ρ" with respect to "σ" is defined by
We see that, when the states are classically related, i.e. "ρσ" = "σρ", the definition coincides with the classical case.
Non-finite (divergent) relative entropy.
In general, the "support" of a matrix "M" is the orthogonal complement of its kernel, i.e. formula_8. When considering the quantum relative entropy, we assume the convention that −"s" · log 0 = ∞ for any "s" > 0. This leads to the definition that
when
This can be interpreted in the following way. Informally, the quantum relative entropy is a measure of our ability to distinguish two quantum states where larger values indicate states that are more different. Being orthogonal represents the most different quantum states can be. This is reflected by non-finite quantum relative entropy for orthogonal quantum states. Following the argument given in the Motivation section, if we erroneously assume the state formula_11 has support in formula_12, this is an error impossible to recover from.
However, one should be careful not to conclude that the divergence of the quantum relative entropy formula_13 implies that the states formula_11 and formula_15 are orthogonal or even very different by other measures. Specifically, formula_13 can diverge when formula_11 and formula_15 differ by a "vanishingly small amount" as measured by some norm. For example, let formula_15 have the diagonal representation
formula_20
with formula_21 for formula_22 and formula_23 for formula_24 where formula_25 is an orthonormal set. The kernel of formula_15 is the space spanned by the set formula_27. Next let
formula_28
for a small positive number formula_29. As formula_11 has support (namely the state formula_31) in the kernel of formula_15, formula_13 is divergent even though the trace norm of the difference formula_34 is formula_35 . This means that difference between formula_11 and formula_15 as measured by the trace norm is vanishingly small as formula_38 even though formula_13 is divergent (i.e. infinite). This property of the quantum relative entropy represents a serious shortcoming if not treated with care.
Non-negativity of relative entropy.
Corresponding classical statement.
For the classical Kullback–Leibler divergence, it can be shown that
and the equality holds if and only if "P" = "Q". Colloquially, this means that the uncertainty calculated using erroneous assumptions is always greater than the real amount of uncertainty.
To show the inequality, we rewrite
Notice that log is a concave function. Therefore -log is convex. Applying Jensen's inequality, we obtain
Jensen's inequality also states that equality holds if and only if, for all "i", "qi" = (∑"qj") "pi", i.e. "p" = "q".
The result.
Klein's inequality states that the quantum relative entropy
is non-negative in general. It is zero if and only if "ρ" = "σ".
Proof
Let "ρ" and "σ" have spectral decompositions
So
Direct calculation gives
where "Pi j" = |"vi*wj"|2.
Since the matrix ("Pi j")"i j" is a doubly stochastic matrix and -log is a convex function, the above expression is
Define "r"i = ∑"j""qj Pi j". Then {"r"i} is a probability distribution. From the non-negativity of classical relative entropy, we have
The second part of the claim follows from the fact that, since -log is strictly convex, equality is achieved in
if and only if ("Pi j") is a permutation matrix, which implies "ρ" = "σ", after a suitable labeling of the eigenvectors {"vi"} and {"wi"}.
Joint convexity of relative entropy.
The relative entropy is jointly convex. For formula_53 and states formula_54 we have
formula_55
Monotonicity of relative entropy.
The relative entropy decreases monotonically under completely positive trace preserving (CPTP) operations formula_56 on density matrices,
formula_57.
This inequality is called Monotonicity of quantum relative entropy.
An entanglement measure.
Let a composite quantum system have state space
and "ρ" be a density matrix acting on "H".
The relative entropy of entanglement of "ρ" is defined by
where the minimum is taken over the family of separable states. A physical interpretation of the quantity is the optimal distinguishability of the state "ρ" from separable states.
Clearly, when "ρ" is not entangled
by Klein's inequality.
Relation to other quantum information quantities.
One reason the quantum relative entropy is useful is that several other important quantum information quantities are special cases of it. Often, theorems are stated in terms of the quantum relative entropy, which lead to immediate corollaries concerning the other quantities. Below, we list some of these relations.
Let "ρ"AB be the joint state of a bipartite system with subsystem "A" of dimension "n"A and "B" of dimension "n"B. Let "ρ"A, "ρ"B be the respective reduced states, and "I"A, "I"B the respective identities. The maximally mixed states are "I"A/"n"A and "I"B/"n"B. Then it is possible to show with direct computation that
where "I"("A":"B") is the quantum mutual information and "S"("B"|"A") is the quantum conditional entropy.