In this paper, we generalize the notions of centroids and barycenters to the broad class of information-theoretic distortion measures called Bregman divergences. Bregman divergences are versatile, and unify quadratic geometric distances with various statistical entropic measures. Because Bregman divergences are typically asymmetric, we consider both the left-sided and right-sided centroids and the symmetrized centroids, and prove that all three are unique. We give closed-form solutions for the sided centroids that are generalized means, and design a provably fast and efficient approximation algorithm for the symmetrized centroid based on its exact geometric characterization that requires solely to walk on the geodesic linking the two sided centroids. We report on our generic implementation for computing entropic centers of image clusters and entropic centers of multivariate normals, and compare our results with former ad-hoc methods.
The symmetrized Kullback-Leibler divergence ( @math -divergence) and symmetrized Itakura-Saito divergence (COSH distance) are often used in sound image applications, where our fast geodesic dichotomic walk algorithm converging to the unique symmetrized Bregman centroid comes in handy over former complex adhoc methods @cite_9 @cite_15 @cite_27 @cite_18 @cite_29 . considers applications of the generic geodesic-walk algorithm to two cases: The symmetrized Kullback-Leibler for probability mass functions represented as @math -dimensional points lying in the @math -dimensional simplex @math . These discrete distributions are handled as multinomials of the exponential families @cite_23 with @math degrees of freedom. We instantiate the generic geodesic-walk algorithm for that setting, show how it compares favorably with the prior convex optimization work of Veldhuis @cite_14 @cite_18 , and validate formally experimental remarks of Veldhuis.
@cite_18 @cite_14 @cite_9 @cite_29 @cite_27 @cite_23 @cite_15
2115685217 2132512523 143794185 2149175990 1887401733 2116327295 2133672829
This paper investigates the use of features based on posterior probabilities of subword units such as phonemes. These features are typically transformed when used as inputs for a hidden Markov model with mixture of Gaussians as emission distribution (HMM GMM). In this work, we introduce a novel acoustic model that avoids the Gaussian assumption and directly uses posterior features without any transformation. This model is described by a finite state machine where each state is characterized by a target distribution and the cost function associated to each state is given by the Kullback-Leibler (KL) divergence between its target distribution and the posterior features. Furthermore, hybrid HMM ANN system can be seen as a particular case of this KL-based model where state target distributions are predefined. A recursive training algorithm to estimate the state target distributions is also presented. This paper discusses the computation of the centroid induced by the symmetrical Kullback-Leibler distance. It is shown that it is the unique zeroing argument of a function which only depends on the arithmetic and the normalized geometric mean of the cluster. An efficient algorithm for its computation is presented. Speech spectra are used as an example. A locking cone chassis and several use methods to eliminate double and triple handling of shipping freight containers. The chassis device comprises two parallel I-beam main rails, a plurality of transverse ribs, a forward cone receiver formed from steel square tubing affixed to a forward end of the main rails, and a similar rear cone receiver affixed to the rear end of the main rails, the only essential difference between the receivers being that the forward cone receiver has an upper flange to guide and to stop a freight container. Each receiver has two box-shaped end portions, which operate like the corner pockets of a freight container-that is, each box-shaped end portion receives a locking cone through an upper surface cone receiving aperture. Each end portion also has access apertures for manually unlocking a cone. Maximum a posteriori (MAP) estimation has been successfully applied to speaker adaptation in speech recognition systems using hidden Markov models. When the amount of data is sufficiently large, MAP estimation yields recognition performance as good as that obtained using maximum-likelihood (ML) estimation. This paper describes a structural maximum a posteriori (SMAP) approach to improve the MAP estimates obtained when the amount of adaptation data is small. A hierarchical structure in the model parameter space is assumed and the probability density functions for model parameters at one level are used as priors for those of the parameters at adjacent levels. Results of supervised adaptation experiments using nonnative speakers' utterances showed that SMAP estimation reduced error rates by 61 when ten utterances were used for adaptation and that it yielded the same accuracy as MAP and ML estimation when the amount of data was sufficiently large. Furthermore, the recognition results obtained in unsupervised adaptation experiments showed that SMAP estimation was effective even when only one utterance from a new speaker was used for adaptation. An effective way to combine rapid supervised adaptation and on-line unsupervised adaptation was also investigated. Concatenative speech synthesis systems attempt to minimize audible signal discontinuities between two successive concatenated units. An objective distance measure which is able to predict audible discontinuities is therefore very important, particularly in unit selection synthesis, for which units are selected from among a large inventory at run time. In this paper, we describe a perceptual test to measure the detection rate of concatenation discontinuity by humans, and then we evaluate 13 different objective distance measures based on their ability to predict the human results. Criteria used to classify these distances include the detection rate, the Bhattacharyya measure of separability of two distributions, and receiver operating characteristic (ROC) curves. Results show that the Kullback-Leibler distance on power spectra has the higher detection rate followed by the Euclidean distance on Mel-frequency cepstral coefficients (MFCC). The Voronoi diagram of a finite set of objects is a fundamental geometric structure that subdivides the embedding space into regions, each region consisting of the points that are closer to a given object than to the others. We may define many variants of Voronoi diagrams depending on the class of objects, the distance functions and the embedding space. In this paper, we investigate a framework for defining and building Voronoi diagrams for a broad class of distance functions called Bregman divergences. Bregman divergences include not only the traditional (squared) Euclidean distance but also various divergence measures based on entropic functions. Accordingly, Bregman Voronoi diagrams allow to define information-theoretic Voronoi diagrams in statistical parametric spaces based on the relative entropy of distributions. We define several types of Bregman diagrams, establish correspondences between those diagrams (using the Legendre transformation), and show how to compute them efficiently. We also introduce extensions of these diagrams, e.g. k-order and k-bag Bregman Voronoi diagrams, and introduce Bregman triangulations of a set of points and their connexion with Bregman Voronoi diagrams. We show that these triangulations capture many of the properties of the celebrated Delaunay triangulation. Finally, we give some applications of Bregman Voronoi diagrams which are of interest in the context of computational geometry and machine learning. The directed divergence, which is a measure based on the discrimination information between two signal classes, is investigated. A simplified expression for computing the directed divergence is derived for comparing two Gaussian autoregressive processes such as those found in speech. This expression alleviates both the computational cost (reduced by two thirds) and the numerical problems encountered in computing the directed divergence. In addition, the simplified expression is compared with the Itakura-Saito distance (which asymptotically approaches the directed divergence). Although the expressions for these two distances closely resemble each other, only moderate correlations between the two were found on a set of actual speech data. >