As mentioned previously, the KL divergence is at the heart of several statistical methods such as logistic regression [1]} and maximum-likelihood (ML) estimation [2]} to fit parametric models of measures.
Given an empirical measure \(_N=\tfrac{1}{N}\sum _{i=1}^N _{x_i}\) , the maximum-likelihood method aims at finding a parameterized probability distribution \(\beta _\)  approximating \(_N\) . The key hypothesis is that \(\beta _\)  has a density \(\xi _\)  with respect to some fixed reference measure \(\mu \) , i.e. \(\beta _= \xi _\mu \)  on \(\) .
The model is estimated on the data support as \(\hat{}_= \sum _{i=1}^N \xi _(x_i)_{x_i}\) .
Maximizing the likelihood is equivalent to minimizing \(\mapsto (_N|\hat{}_)\)  with respect to the parameter \(\) .
Logistic regression or Gaussian mixture models [3]} correspond to the above minimization problem with a particular choice of the density \(\xi _\) .
These minimization problems are often challenging because of the non-convexity w.r.t \(\) .
A standard algorithm is the expectation-maximization (EM) algorithm [4]}, which is sensitive to the initialization and can be trapped in local minima.
