The recently developed bag-of-paths framework consists in setting a Gibbs-Boltzmann distribution on all feasible paths of a graph. This probability distribution favors short paths over long ones, with a free parameter (the temperature @math ) controlling the entropic level of the distribution. This formalism enables the computation of new distances or dissimilarities, interpolating between the shortest-path and the resistance distance, which have been shown to perform well in clustering and classification tasks. In this work, the bag-of-paths formalism is extended by adding two independent equality constraints fixing starting and ending nodes distributions of paths. When the temperature is low, this formalism is shown to be equivalent to a relaxation of the optimal transport problem on a network where paths carry a flow between two discrete distributions on nodes. The randomization is achieved by considering free energy minimization instead of traditional cost minimization. Algorithms computing the optimal free energy solution are developed for two types of paths: hitting (or absorbing) paths and non-hitting, regular paths, and require the inversion of an @math matrix with @math being the number of nodes. Interestingly, for regular paths, the resulting optimal policy interpolates between the deterministic optimal transport policy ( @math ) and the solution to the corresponding electrical circuit ( @math ). Two distance measures between nodes and a dissimilarity between groups of nodes, both integrating weights on nodes, are derived from this framework.
 Optimal transport distances are a fundamental family of distances for probability measures and histograms of features. Despite their appealing theoretical properties, excellent performance in retrieval tasks and intuitive formulation, their computation involves the resolution of a linear program whose cost can quickly become prohibitive whenever the size of the support of these measures or the histograms' dimension exceeds a few hundred. We propose in this work a new family of optimal transport distances that look at transport problems from a maximum-entropy perspective. We smooth the classic optimal transport problem with an entropic regularization term, and show that the resulting optimum is also a distance which can be computed through Sinkhorn's matrix scaling algorithm at a speed that is several orders of magnitude faster than that of transport solvers. We also show that this regularized distance improves upon classic optimal transport distances on the MNIST classification problem. This article introduces a generalization of discrete Optimal Transport that includes a regularity penalty and a relaxation of the bijectivity constraint. The corresponding transport plan is solved by minimizing an energy which is a convexification of an integer optimization problem. We propose to use a proximal splitting scheme to perform the minimization on large scale imaging problems. For un-regularized relaxed transport, we show that the relaxation is tight and that the transport plan is an assignment. In the general case, the regularization prevents the solution from being an assignment, but we show that the corresponding map can be used to solve imaging problems. We show an illustrative application of this discrete regularized transport to color transfer between images. This imaging problem cannot be solved in a satisfying manner without relaxing the bijective assignment constraint because of mass variation across image color palettes. Furthermore, the regularization of the transport plan helps remove colorization artifacts due to noise amplification. Preface. 1. Introduction to Entropy and Entropy Optimization Principles. 2. Entropy Optimization Models. 3. Entropy Optimization Methods: Linear Case. 4. Entropy Optimization Methods: General Convex Case. 5. Entropic Perturbation Approach to Mathematical Programming. 6. Lp-Norm Perturbation Approach: A Generalization of Entropic Perturbation. 7. Extensions and Related Results. Bibliography. Index.   Entropy optimization principles Jaynes' maximum entropy principle applications of Jaynes' maximum entropy principle Kullback's minimum cross-entropy principle further applications of MaxEnt and MinEnt new entropy optimization principles generalized principles of maximum entropy the four inverse maximum entropy principles. Partial table of contents: Maximum--Entropy Probability Distributions: Principles, Formalism and Techniques. Maximum--Entropy Discrete Univariate Probability Distributions. Maximum--Entropy Discrete Multivariate Probability Distributions. Maximum--Entropy Continuous Multivariate Probability Distributions. Maximum--Entropy Distributions in Statistical Mechanics. Minimum Discrepancy Measures. Concavity (Convexity) of Maximum--Entropy (Minimum Information) Functions. Equivalence of Maximum--Entropy Principle and Gauss's Principle of Density Estimation. Maximum--Entropy Principle and Contingency Tables. Maximum--Entropy Principle and Statistics. Maximum--Entropy Models in Regional and Urban Planning. Maximum--Entropy Models in Marketing and Elections. Maximum--Entropy Spectral Analysis. Maximum--Entropy Image Reconstruction. Maximum--Entropy Principle in Operations Research. References. Author Index. Subject Index. Probability distributions and histograms are natural representations for product ratings, traffic measurements, and other data considered in many machine learning applications. Thus, this paper introduces a technique for graph-based semi-supervised learning of histograms, derived from the theory of optimal transportation. Our method has several properties making it suitable for this application; in particular, its behavior can be characterized by the moments and shapes of the histograms at the labeled nodes. In addition, it can be used for histograms on non-standard domains like circles, revealing a strategy for manifold-valued semi-supervised learning. We also extend this technique to related problems such as smoothing distributions on graph nodes. Domain adaptation is one of the most challenging tasks of modern data analytics. If the adaptation is done correctly, models built on a specific data representation become more robust when confronted to data depicting the same classes, but described by another observation system. Among the many strategies proposed, finding domain-invariant representations has shown excellent properties, in particular since it allows to train a unique classifier effective in all domains. In this paper, we propose a regularized unsupervised optimal transportation model to perform the alignment of the representations in the source and target domains. We learn a transportation plan matching both PDFs, which constrains labeled samples of the same class in the source domain to remain close during transport. This way, we exploit at the same time the labeled samples in the source and the distributions observed in both domains. Experiments on toy and challenging real visual adaptation examples show the interest of the method, that consistently outperforms state of the art approaches. In addition, numerical experiments show that our approach leads to better performances on domain invariant deep learning features and can be easily adapted to the semi-supervised case where few labeled samples are available in the target domain.