We study properties of an attractive-repulsive energy functional based on power-kernels, which can be used for halftoning of images. In the first part of this work, using a variational framework for probability measures, we examine existence and behavior of minimizers to the functional and to a regularization of it by a total variation term. Moreover, we introduce particle approximations to the functional and to its regularized version and prove their consistency in terms of Gamma-convergence, which we additionally illustrate by numerical examples. In the second part, we consider the gradient flow of the functional in the 2-Wasserstein space and prove statements about its asymptotic behavior for large times, for which we employ the pseudo-inverse technique for probability measures in 1D. Depending on the parameter range, this includes existence of a subsequence converging to a steady state or even convergence of the whole trajectory to a limit which we can specify explicitly. For both parts of the work, a key ingredient is the generalized Fourier transform, which allows us to verify the conditional positive definiteness of the interaction kernel for coinciding attractive and repulsive exponents.
With respect to our particular problem and the static setting, see @cite_28 for efficient optimization algorithms to find local minima of ( E ) and @cite_6 for the relationship of minimizers of ( E ) and the error of quadrature formulas, also highlighting the connection between those minimizers and the problem of optimal quantization of measures (see @cite_23 @cite_32 ). As for the gradient flow, see for example @cite_3 for the analysis of symmetric steady states for the gradient flow of interaction functionals similar to ( W ), but being composed of the sum of an attractive and a repulsive power function.
@cite_28 @cite_32 @cite_3 @cite_6 @cite_23
2655474054 2146989110 2272870368 2962851402
We focus on nonconvex and nonsmooth minimization problems with a composite objective, where the differentiable part of the objective is freed from the usual and restrictive global Lipschitz gradient continuity assumption. This longstanding smoothness restriction is pervasive in first order methods (FOM), and was recently circumvent for convex composite optimization by Bauschke, Bolte and Teboulle, through a simple and elegant framework which captures, all at once, the geometry of the function and of the feasible set. Building on this work, we tackle genuine nonconvex problems. We first complement and extend their approach to derive a full extended descent lemma by introducing the notion of smooth adaptable functions. We then consider a Bregman-based proximal gradient methods for the nonconvex composite model with smooth adaptable functions, which is proven to globally converge to a critical point under natural assumptions on the problem's data. To illustrate the power and potential of our general framework and results, we consider a broad class of quadratic inverse problems with sparsity constraints which arises in many fundamental applications, and we apply our approach to derive new globally convergent schemes for this class. A central challenge to many fields of science and engineering involves minimizing non-convex error functions over continuous, high dimensional spaces. Gradient descent or quasi-Newton methods are almost ubiquitously used to perform such minimizations, and it is often thought that a main source of difficulty for these local methods to find the global minimum is the proliferation of local minima with much higher error than the global minimum. Here we argue, based on results from statistical physics, random matrix theory, neural network theory, and empirical evidence, that a deeper and more profound difficulty originates from the proliferation of saddle points, not local minima, especially in high dimensional problems of practical interest. Such saddle points are surrounded by high error plateaus that can dramatically slow down learning, and give the illusory impression of the existence of a local minimum. Motivated by these arguments, we propose a new approach to second-order optimization, the saddle-free Newton method, that can rapidly escape high dimensional saddle points, unlike gradient descent and quasi-Newton methods. We apply this algorithm to deep or recurrent neural network training, and provide numerical evidence for its superior optimization performance. We consider the problem of online scheduling of jobs on unrelated machines with dynamic speed scaling to minimize the sum of energy and weighted flow time. We give an algorithm with an almost optimal competitive ratio for arbitrary power functions. (No earlier results handled arbitrary power functions for minimizing flow time plus energy with unrelated machines.) For power functions of the form f(s) = sα for some constant α > 1, we get a competitive ratio of O(α log α), improving upon a previous competitive ratio of O(α2) by [3], along with a matching lower bound of Ω(α log α). Further, in the resource augmentation model, with a 1 + e speed up, we give a 2(1 e + 1) competitive algorithm, with essentially the same techniques, improving the bound of 1 + O(1 e2) by [15] and matching the bound of [3] for the special case of fixed speed unrelated machines. Unlike the previous results most of which used an amortized local competitiveness argument or dual fitting methods, we use a primal-dual method, which is useful not only to analyze the algorithms but also to design the algorithm itself. We consider the fundamental problem in nonconvex optimization of efficiently reaching a stationary point. In contrast to the convex case, in the long history of this basic problem, the only known theoretical results on first-order nonconvex optimization remain to be full gradient descent that converges in O(1 e) iterations for smooth objectives, and stochastic gradient descent that converges in O(1 e2) iterations for objectives that are sum of smooth functions. We provide the first improvement in this line of research. Our result is based on the variance reduction trick recently introduced to convex optimization, as well as a brand new analysis of variance reduction that is suitable for non-convex optimization. For objectives that are sum of smooth functions, our first-order minibatch stochastic method converges with an O(1 e) rate, and is faster than full gradient descent by Ω(n1 3). We demonstrate the effectiveness of our methods on empirical risk minimizations with nonconvex loss functions and training neural nets.