Continuous-time projected dynamical systems are an elementary class of discontinuous dynamical systems with trajectories that remain in a feasible domain by means of projecting outward-pointing vector fields. They are essential when modeling physical saturation, constraints of motion, as well as studying projection-based numerical optimization algorithms. Motivated by the emerging application of feedback-based continuous-time optimization schemes that rely on the physical system to enforce nonlinear hard constraints, we study the fundamental properties of these dynamics on general locally-Euclidean sets. Among others, we propose the use of Krasovskii solutions, show their existence on nonconvex, irregular subsets of low-regularity Riemannian manifolds, and investigate how they relate to conventional Carath 'eodory solutions. Furthermore, we establish conditions for uniqueness, thereby introducing a generalized definition of prox-regularity which is suitable for non-flat domains. Finally, we use these results to study the stability and convergence of projected gradient flows as an illustrative application of our framework. We provide simple counter-examples for our main results to illustrate the necessity of our already weak assumptions.
Assuming that @math is the gradient field of a potential function and @math is Clarke regular, we can show that the connection between projected gradients and subgradients which is well-known for convex functions (and lesser known for regular functions @cite_45 @cite_29 ) generalizes to a variable metric. For this, recall that @math where @math is open and @math is if its epigraph @math is Clarke regular.
@cite_29 @cite_45
2766371994 2053661713 2963116072 1945101555
We consider the problem of learning a one-hidden-layer neural network: we assume the input @math is from Gaussian distribution and the label @math , where @math is a nonnegative vector in @math with @math , @math is a full-rank weight matrix, and @math is a noise vector. We first give an analytic formula for the population risk of the standard squared loss and demonstrate that it implicitly attempts to decompose a sequence of low-rank tensors simultaneously. Inspired by the formula, we design a non-convex objective function @math whose landscape is guaranteed to have the following properties: 1. All local minima of @math are also global minima. 2. All global minima of @math correspond to the ground truth parameters. 3. The value and gradient of @math can be estimated using samples. With these properties, stochastic gradient descent on @math provably converges to the global minimum and learn the ground-truth parameters. We also prove finite sample complexity result and validate the results by simulations. The main result of the paper consists of the theorem that under certain, natural assumptions the local conditional maximum @math of the function f on the set [ A = x R^n | _i (x) 0, _j (x) = 0,i = 1, ,k,j = 1, ,l ] is identical with the unconditional maximum of the potential function [ p(x, ) = f(x) + _ i = 1 ^k neg ( _i (x)) - _ j = 1 ^l | (x) | , x R^n , 0, ] for @math sufficiently small. There is also provided a draft of a modified gradient procedure for maximizing the potential @math since it is generally nonsmooth even for differentiable f, @math and @math . In this paper, we study the problem of maximizing continuous submodular functions that naturally arise in many learning applications such as those involving utility functions in active learning and sensing, matrix approximations and network inference. Despite the apparent lack of convexity in such functions, we prove that stochastic projected gradient methods can provide strong approximation guarantees for maximizing continuous submodular functions with convex constraints. More specifically, we prove that for monotone continuous DR-submodular functions, all fixed points of projected gradient ascent provide a factor @math approximation to the global maxima. We also study stochastic gradient methods and show that after @math iterations these methods reach solutions which achieve in expectation objective values exceeding @math . An immediate application of our results is to maximize submodular functions that are defined stochastically, i.e. the submodular function is defined as an expectation over a family of submodular functions with an unknown distribution. We will show how stochastic gradient methods are naturally well-suited for this setting, leading to a factor @math approximation when the function is monotone. In particular, it allows us to approximately maximize discrete, monotone submodular optimization problems via projected gradient ascent on a continuous relaxation, directly connecting the discrete and continuous domains. Finally, experiments on real data demonstrate that our projected gradient methods consistently achieve the best utility compared to other continuous baselines while remaining competitive in terms of computational effort. Let @math be a smooth scheme over an algebraically closed field @math of characteristic zero and @math a regular function, and write @math Crit @math , as a closed subscheme of @math . The motivic vanishing cycle @math is an element of the @math -equivariant motivic Grothendieck ring @math defined by Denef and Loeser math.AG 0006050 and Looijenga math.AG 0006220, and used in Kontsevich and Soibelman's theory of motivic Donaldson-Thomas invariants, arXiv:0811.2435. We prove three main results: (a) @math depends only on the third-order thickenings @math of @math . (b) If @math is another smooth scheme, @math is regular, @math Crit @math , and @math is an embedding with @math and @math an isomorphism, then @math equals @math "twisted" by a motive associated to a principal @math -bundle defined using @math , where now we work in a quotient ring @math of @math . (c) If @math is an "oriented algebraic d-critical locus" in the sense of Joyce arXiv:1304.4508, there is a natural motive @math , such that if @math is locally modelled on Crit @math , then @math is locally modelled on @math . Using results from arXiv:1305.6302, these imply the existence of natural motives on moduli schemes of coherent sheaves on a Calabi-Yau 3-fold equipped with "orientation data", as required in Kontsevich and Soibelman's motivic Donaldson-Thomas theory arXiv:0811.2435, and on intersections of oriented Lagrangians in an algebraic symplectic manifold. This paper is an analogue for motives of results on perverse sheaves of vanishing cycles proved in arXiv:1211.3259. We extend this paper to Artin stacks in arXiv:1312.0090.