In a graph convolutional network, we assume that the graph @math is generated with respect to some observation noise. We make small random perturbations @math of the graph and try to improve generalization. Based on quantum information geometry, we can have quantitative measurements on the scale of @math . We try to maximize the intrinsic scale of the permutation with a small budget while minimizing the loss based on the perturbed @math . Our proposed model can consistently improve graph convolutional networks on semi-supervised node classification tasks with reasonable computational overhead. We present two different types of geometry on the manifold of graphs: one is for measuring the intrinsic change of a graph; the other is for measuring how such changes can affect externally a graph neural network. These new analytical tools will be useful in developing a good understanding of graph neural networks and fostering new techniques.
Tools from quantum information geometry are applied to machine learning @cite_15 @cite_5 but not yet ported to the domain of graph neural networks. In information geometry one can have different matrix divergences @cite_39 that can be applied on p.s.d. matrices. We point the reader to related definitions of the discrete Fisher information @cite_12 without illuminate the details.
@cite_5 @cite_15 @cite_12 @cite_39
2964259376 2964068697 2942681667 597395834
Embedding complex objects as vectors in low dimensional spaces is a longstanding problem in machine learning. We propose in this work an extension of that approach, which consists in embedding objects as elliptical probability distributions, namely distributions whose densities have elliptical level sets. We endow these measures with the 2-Wasserstein metric, with two important benefits: For such measures, the squared 2-Wasserstein metric has a closed form, equal to the sum of the squared Euclidean distance between means and the squared Bures metric between covariance matrices. The latter is a Riemannian metric between positive semi-definite matrices, which turns out to be Euclidean on a suitable factor representation of such matrices, which is valid on the entire geodesic between these matrices. The 2-Wasserstein distance boils down to the usual Euclidean metric when comparing Diracs, and therefore provides the natural framework to extend point embeddings. We show that for these reasons Wasserstein elliptical embeddings are more intuitive and yield tools that are better behaved numerically than the alternative choice of Gaussian embeddings with the Kullback-Leibler divergence. In particular, and unlike previous work based on the KL geometry, we learn elliptical distributions that are not necessarily diagonal. We demonstrate the interest of elliptical embeddings by using them for visualization, to compute embeddings of words, and to reflect entanglement or hypernymy. Abstract The metric d ( A , B ) = tr A + tr B − 2 tr ( A 1 ∕ 2 B A 1 ∕ 2 ) 1 ∕ 2 1 ∕ 2 on the manifold of n × n positive definite matrices arises in various optimisation problems, in quantum information and in the theory of optimal transport. It is also related to Riemannian geometry. In the first part of this paper we study this metric from the perspective of matrix analysis, simplifying and unifying various proofs. Then we develop a theory of a mean of two, and a barycentre of several, positive definite matrices with respect to this metric. We explain some recent work on a fixed point iteration for computing this Wasserstein barycentre. Our emphasis is on ideas natural to matrix analysis. Existing popular methods for semi-supervised learning with Graph Neural Networks (such as the Graph Convolutional Network) provably cannot learn a general class of neighborhood mixing relationships. To address this weakness, we propose a new model, MixHop, that can learn these relationships, including difference operators, by repeatedly mixing feature representations of neighbors at various distances. Mixhop requires no additional memory or computational complexity, and outperforms on challenging baselines. In addition, we propose sparsity regularization that allows us to visualize how the network prioritizes neighborhood information across different graph datasets. Our analysis of the learned architectures reveals that neighborhood mixing varies per datasets. Book dedication.- Preface.- Part I: State-of-the-art surveys & original matrix theory work.- Part II: Advanced matrix theory for radar processing.- Part III: Matrix-based signal processing applications.- Index of terms