For exponential-family distributions, the Kullback-Leibler (KL) divergence is a Bregman divergence [1]}, which is better indicated to optimize the parameters of a probability distribution [2]}:
\(\mathbb {D}(\theta ,\theta _t) = \mathbb {D}_{\sf KL}(p_\theta \,||\, p_{\theta _t}) = \mathbb {E}_{P\sim p_\theta } \Big \lbrace  \log \frac{p_\theta (P)}{p_{\theta _t}(P)} \Big \rbrace .\) 
