In this paper, we generalize the notions of centroids and barycenters to the broad class of information-theoretic distortion measures called Bregman divergences. Bregman divergences are versatile, and unify quadratic geometric distances with various statistical entropic measures. Because Bregman divergences are typically asymmetric, we consider both the left-sided and right-sided centroids and the symmetrized centroids, and prove that all three are unique. We give closed-form solutions for the sided centroids that are generalized means, and design a provably fast and efficient approximation algorithm for the symmetrized centroid based on its exact geometric characterization that requires solely to walk on the geodesic linking the two sided centroids. We report on our generic implementation for computing entropic centers of image clusters and entropic centers of multivariate normals, and compare our results with former ad-hoc methods.
In section , we show that the two sided Bregman centroids @math and @math with respect to Bregman divergence @math are unique and easily obtained as generalized means for the identity and @math functions, respectively. We extend Sibson' s notion of information radius @cite_28 for these sided centroids, and show that they are both equal to the @math -Jensen difference, a generalized Jensen-Shannon divergence @cite_7 also known as Burbea-Rao divergences @cite_19 .
@cite_28 @cite_19 @cite_7
2070134780 2146950091
Three measures of divergence between vectors in a convex set of a n -dimensional real vector space are defined in terms of certain types of entropy functions, and their convexity property is studied. Among other results, a classification of the entropies of degree is obtained by the convexity of these measures. These results have applications in information theory and biological studies. A novel class of information-theoretic divergence measures based on the Shannon entropy is introduced. Unlike the well-known Kullback divergences, the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved. More importantly, their close relationship with the variational distance and the probability of misclassification error are established in terms of bounds. These bounds are crucial in many applications of divergence measures. The measures are also well characterized by the properties of nonnegativity, finiteness, semiboundedness, and boundedness. >