The KL divergence is also at the heart of information geometry [1]}, which is an alternative way to define distances between distributions using Riemannian geometry.
It consists in defining a canonical Riemannian manifold structure on a parametric family \((\beta _\theta )_\theta \)  of distributions.
In the case of distributions with parameters \(\in ^d\) , if all \(\beta _\theta \)  have a density \(\xi _\theta \)  with respect to a fixed reference measure, then the Riemannian metric is the second order Taylor expansion at 0 of the function \(\delta \mapsto (_|_{+})\)  w.r.t. \(\delta \) .
The second order term is called the Fisher information matrix and reads
\(() \triangleq \int _\big [\nabla \log \xi _\theta (x)\nabla \log \xi _\theta (x)^\top \big ]\beta _\theta (x) \in ^{d\times d}.\) 
