In a graph convolutional network, we assume that the graph @math is generated with respect to some observation noise. We make small random perturbations @math of the graph and try to improve generalization. Based on quantum information geometry, we can have quantitative measurements on the scale of @math . We try to maximize the intrinsic scale of the permutation with a small budget while minimizing the loss based on the perturbed @math . Our proposed model can consistently improve graph convolutional networks on semi-supervised node classification tasks with reasonable computational overhead. We present two different types of geometry on the manifold of graphs: one is for measuring the intrinsic change of a graph; the other is for measuring how such changes can affect externally a graph neural network. These new analytical tools will be useful in developing a good understanding of graph neural networks and fostering new techniques.
Our work is related in that we also construct FIM for the purpose of extrinsically parameterize the graph Laplacian. We perform a maximization these parameters to flatten the FIM around its local optimum which corresponds to decreasing eigenvalues in @cite_19 , thus making our approach well regularized in the sense of flattening the largest curvature associated with FIM (improved smoothness). With the smoothness, the classification performance typically degrades, see the impact of smoothness on kernel representations @cite_1 . Indeed, study @cite_6 further shows there is a fundamental trade-off between high accuracy and the adversarial robustness.
@cite_19 @cite_1 @cite_6
2963470399 2000214666 2964040595 2962720813
The Fisher information matrix (FIM) is a fundamental quantity to represent the characteristics of a stochastic model, including deep neural networks (DNNs). The present study reveals novel statistics of FIM that are universal among a wide class of DNNs. To this end, we use random weights and large width limits, which enables us to utilize mean field theories. We investigate the asymptotic statistics of the FIM’s eigenvalues and reveal that most of them are close to zero while the maximum eigenvalue takes a huge value. Because the landscape of the parameter space is defined by the FIM, it is locally flat in most dimensions, but strongly distorted in others. Moreover, we demonstrate the potential usage of the derived statistics in learning strategies. First, small eigenvalues that induce flatness can be connected to a norm-based capacity measure of generalization ability. Second, the maximum eigenvalue that induces the distortion enables us to quantitatively estimate an appropriately sized learning rate for gradient methods to converge. In this paper, we develop methods to rapidly remove rough features from irregularly triangulated data intended to portray a smooth surface. The main task is to remove undesirable noise and uneven edges while retaining desirable geometric features. The problem arises mainly when creating high-fidelity computer graphics objects using imperfectly-measured data from the real world. Our approach contains three novel features: an implicit integration method to achieve efficiency, stability, and large time-steps; a scale-dependent Laplacian operator to improve the diffusion process; and finally, a robust curvature flow operator that achieves a smoothing of the shape itself, distinct from any parameterization. Additional features of the algorithm include automatic exact volume preservation, and hard and soft constraints on the positions of the points in the mesh. We compare our method to previous operators and related algorithms, and prove that our curvature and Laplacian operators have several mathematically-desirable qualities that improve the appearance of the resulting surface. In consequence, the user can easily select the appropriate operator according to the desired type of fairing. Finally, we provide a series of examples to graphically and numerically demonstrate the quality of our results. Abstract Nonlinear dimensionality reduction embeddings computed from datasets do not provide a mechanism to compute the inverse map. In this paper, we address the problem of computing a stable inverse map to such a general bi-Lipschitz map. Our approach relies on radial basis functions (RBFs) to interpolate the inverse map everywhere on the low-dimensional image of the forward map. We demonstrate that the scale-free cubic RBF kernel performs better than the Gaussian kernel: it does not suffer from ill-conditioning, and does not require the choice of a scale. The proposed construction is shown to be similar to the Nystrom extension of the eigenvectors of the symmetric normalized graph Laplacian matrix. Based on this observation, we provide a new interpretation of the Nystrom extension with suggestions for improvement. Given a weighted graph with N vertices, consider a real-valued regression problem in a semisupervised setting, where one observes n labeled vertices, and the task is to label the remaining ones. We present a theoretical study of p-based Laplacian regularization under a d-dimensional geometric random graph model. We provide a variational characterization of the performance of this regularized learner as N grows to infinity while n stays constant; the associated optimality conditions lead to a partial differential equation that must be satisfied by the associated function estimate f . From this formulation we derive several predictions on the limiting behavior the function f , including (a) a phase transition in its smoothness at the threshold p = d + 1; and (b) a tradeoff between smoothness and sensitivity to the underlying unlabeled data distribution P . Thus, over the range p ≤ d, the function estimate f is degenerate and “spiky,” whereas for p ≥ d + 1, the function estimate f is smooth. We show that the effect of the underlying density vanishes monotonically with p, such that in the limit p = ∞, corresponding to the so-called Absolutely Minimal Lipschitz Extension, the estimate f is independent of the distribution P . Under the assumption of semi-supervised smoothness, ignoring P can lead to poor statistical performance; in particular, we construct a specific example for d = 1 to demonstrate that p = 2 has lower risk than p =∞ due to the former penalty adapting to P and the latter ignoring it. We also provide simulations that verify the accuracy of our predictions for finite sample sizes. Together, these properties show that p = d + 1 is an optimal choice, yielding a function estimate f that is both smooth and non-degenerate, while remaining maximally sensitive to P .