We consider point clouds obtained as random samples of a measure on a Euclidean domain. A graph representing the point cloud is obtained by assigning weights to edges based on the distance between the points they connect. Our goal is to develop mathematical tools needed to study the consistency, as the number of available data points increases, of graph-based machine learning algorithms for tasks such as clustering. In particular, we study when the cut capacity, and more generally total variation, on these graphs is a good approximation of the perimeter (total variation) in the continuum setting. We address this question in the setting of Γ-convergence. We obtain almost optimal conditions on the scaling, as the number of points increases, of the size of the neighborhood over which the points are connected by an edge for the Γ-convergence to hold. Taking of the limit is enabled by a transportation based metric which allows us to suitably compare functionals defined on different point clouds.
In the discrete setting, works related to the @math -convergence of functionals to continuous functionals involving perimeter include @cite_44 , @cite_55 and @cite_36 . The results by Braides and Yip @cite_44 , can be interpreted as the analogous results in a discrete setting to the ones obtained by Modica and Mortola. They give the description of the limiting functional (in the sense of @math -convergence) after appropriately rescaling the energies. In the discretized version considered, they work on a regular grid and the gradient term gets replaced by a finite-difference approximation that depends on the mesh size @math . Van Gennip and Bertozzi @cite_55 consider a similar problem and obtain analogous results. In @cite_36 , Chambolle, Giacomini and Lussardi consider a very general class of anisotropic perimeters defined on discrete subsets of a finite lattice of the form @math . They prove the @math -convergence of the functionals as @math to an anisotropic perimeter defined on a given domain in @math .
@cite_44 @cite_55 @cite_36
2129572061 2463117017 2949304946 2069278600 2531021157 2129732816 2086002908 2107342718 1967138577 1804927206 2655474054 1992225844 2003666784 2738326163 2609098951 1922672875 2015836240 2338955270 2963833097 2277973662
We consider a class of discrete convex functionals which satisfy a (generalized) coarea formula. These functionals, based on submodular interactions, arise in discrete optimization and are known as a large class of problems which can be solved in polynomial time. In particular, some of them can be solved very efficiently by maximal flow algorithms and are quite popular in the image processing community. We study the limit in the continuum of these functionals, show that they always converge to some "crystalline" perimeter total variation, and provide an almost explicit formula for the limiting functional. Approximations of solutions of fractional Laplacian equations on bounded domains are considered. Such equations allow global interactions between points separated by arbitrarily large distances. Two approximations are introduced. First, interactions are localized so that only points less than some specified distance, referred to as the interaction radius, are allowed to interact. The resulting truncated problem is a special case of a more general nonlocal diffusion problem. The second approximation is the spatial discretization of the related nonlocal diffusion problem. A recently developed abstract framework for asymptotically compatible schemes is applied to prove convergence results for solutions of the truncated and discretized problem to the solutions of the fractional Laplacian problems. Intermediate results also provide new convergence results for the nonlocal diffusion problem. Special attention is paid to limiting behaviors as the interaction radius increases and the spatial grid size decreases, regardless of how these parameters may or may not be dependent. In particular, we show that conforming Galerkin finite element approximations of the nonlocal diffusion equation are always asymptotically compatible schemes for the corresponding fractional Laplacian model as the interaction radius increases and the grid size decreases. The results are developed with minimal regularity assumptions on the solution and are applicable to general domains and general geometric meshes with no restriction on the space dimension and with data that are only required to be square integrable. Furthermore, our results also solve an open conjecture given in the literature about the convergence of numerical solutions on a fixed mesh as the interaction radius increases. Given a discrete random walk on a finite graph @math , the vacant set and vacant net are, respectively, the sets of vertices and edges which remain unvisited by the walk at a given step @math . These sets induce subgraphs of the underlying graph. Let @math be the subgraph of @math induced by the vacant set of the walk at step @math . Similarly, let @math be the subgraph of @math induced by the edges of the vacant net. For random @math -regular graphs @math , it was previously established that for a simple random walk, the graph @math of the vacant set undergoes a phase transition in the sense of the phase transition on Erd os -Renyi graphs @math . Thus, for @math there is an explicit value @math of the walk, such that for @math , @math has a unique giant component, plus components of size @math , whereas for @math all the components of @math are of size @math . We establish the threshold value @math for a phase transition in the graph @math of the vacant net of a simple random walk on a random @math -regular graph. We obtain the corresponding threshold results for the vacant set and vacant net of two modified random walks. These are a non-backtracking random walk, and, for @math even, a random walk which chooses unvisited edges whenever available. This allows a direct comparison of thresholds between simple and modified walks on random @math -regular graphs. The main findings are the following: As @math increases the threshold for the vacant set converges to @math in all three walks. For the vacant net, the threshold converges to @math for both the simple random walk and non-backtracking random walk. When @math is even, the threshold for the vacant net of the unvisited edge process converges to @math , which is also the vertex cover time of the process. We present a new efficient algorithm for the search version of the approximate Closest Vector Problem with Preprocessing (CVPP). Our algorithm achieves an approximation factor of O(n sqrt log n ), improving on the previous best of O(n^ 1.5 ) due to Lag arias, Lenstra, and Schnorr hkzbabai . We also show, somewhat surprisingly, that only O(n) vectors of preprocessing advice are sufficient to solve the problem (with the slightly worse approximation factor of O(n)). We remark that this still leaves a large gap with respect to the decisional version of CVPP, where the best known approximation factor is O(sqrt n log n ) due to Aharonov and Regev AharonovR04 . To achieve these results, we show a reduction to the same problem restricted to target points that are close to the lattice and a more efficient reduction to a harder problem, Bounded Distance Decoding with preprocessing (BDDP). Combining either reduction with the previous best-known algorithm for BDDP by Liu, Lyubashevsky, and Micciancio LiuLM06 gives our main result. In the setting of CVP without preprocessing, we also give a reduction from (1+eps)gamma approximate CVP to gamma approximate CVP where the target is at distance at most 1+1 eps times the minimum distance (the length of the shortest non-zero vector) which relies on the lattice sparsification techniques of Dadush and Kun DadushK13 . As our final and most technical contribution, we present a substantially more efficient variant of the LLM algorithm (both in terms of run-time and amount of preprocessing advice), and via an improved analysis, show that it can decode up to a distance proportional to the reciprocal of the smoothing parameter of the dual lattice MR04 . We show that this is never smaller than the LLM decoding radius, and that it can be up to an wide tilde Omega (sqrt n ) factor larger. In our LICS 2004 paper we introduced an approach to the study of the local structure of finite algebras and relational structures that aims at applications in the Constraint Satisfaction Problem (CSP). This approach involves a graph associated with an algebra @math or a relational structure A, whose vertices are the elements of @math (or A), the edges represent subsets of @math such that the restriction of some term operation of @math is ‘good’ on the subset, that is, act as an operation of one of the 3 types: semilattice, majority, or affine. In this paper we significantly refine and advance this approach. In particular, we prove certain connectivity and rectangularity properties of relations over algebras related to components of the graph connected by semilattice and affine edges. We also prove a result similar to 2-decomposition of relations invariant under a majority operation, only here we do not impose any restrictions on the relation. These results allow us to give a new, somewhat more intuitive proof of the bounded width theorem: the CSP over algebra @math has bounded width if and only if @math does not contain affine edges. Actually, this result shows that bounded width implies width (2,3). We also consider algebras with edges from a restricted set of types. In particular, it can be proved that type restrictions are preserved under the standard algebraic constructions. Finally, we prove that algebras without semilattice edges have few subalgebras of powers, that is, the CSP over such algebras is also polynomial time. We study the convergence properties of an alternating proximal minimization algorithm for nonconvex structured functions of the type: L(x,y)=f(x)+Q(x,y)+g(y), where f and g are proper lower semicontinuous functions, defined on Euclidean spaces, and Q is a smooth function that couples the variables x and y. The algorithm can be viewed as a proximal regularization of the usual Gauss-Seidel method to minimize L. We work in a nonconvex setting, just assuming that the function L satisfies the Kurdyka-Łojasiewicz inequality. An entire section illustrates the relevancy of such an assumption by giving examples ranging from semialgebraic geometry to “metrically regular” problems. Our main result can be stated as follows: If L has the Kurdyka-Łojasiewicz property, then each bounded sequence generated by the algorithm converges to a critical point of L. This result is completed by the study of the convergence rate of the algorithm, which depends on the geometrical properties of the function L around its critical points. When specialized to @math and to f, g indicator functions, the algorithm is an alternating projection mehod (a variant of von Neumann's) that converges for a wide class of sets including semialgebraic and tame sets, transverse smooth manifolds or sets with “regular” intersection. To illustrate our results with concrete problems, we provide a convergent proximal reweighted l1 algorithm for compressive sensing and an application to rank reduction problems. We investigate the limiting description for a finite-difference approximation of a singularly perturbed Allen--Cahn type energy functional. The key issue is to understand the interaction between two small length-scales: the interfacial thickness @math and the mesh size of spatial discretization @math . Depending on their relative sizes, we obtain results in the framework of @math -convergence for the (i) subcritical ( @math ), (ii) critical ( @math ), and (iii) supercritical ( @math ) cases. The first case leads to the same area functional as the spatially continuous case while the third gives the same result as that coming from a ferromagnetic spin energy. The critical case can be regarded as an interpolation between the two. We define a natural notion of efficiency for approximate nearest-neighbor (ANN) search in general n-point metric spaces, namely the existence of a randomized algorithm which answers (1 + e)-ANN queries in polylog(n) time using only polynomial space. We then study which families of metric spaces admit efficient ANN schemes in the black-box model, where only oracle access to the distance function is given, and any query consistent with the triangle inequality may be asked.For e < 2 5, we offer a complete answer to this problem. Using the notion of metric dimension defined in [A. Gupta, et al, Bounded geometries, fractals, and low-distortion embeddings, in: 44th Annu. IEEE Symp. on Foundations of Computer Science, 2003, pp. 534-543] (a la [P. Assouad, Plongements lipschitziens dans Rn, Bull. Soc. Math. France 111 (4) (1983) 429-448]), we show that a metric space X admits an efficient (1+e)-ANN scheme for any e < 2 5 if and only if dim(X) = O(log log n). For coarser approximations, clearly the upper bound continues to hold, but there is a threshold at which our lower bound breaks down--this is precisely when points in the "ambient space" may begin to affect the complexity of "hard" subspaces S ⊆ X. Indeed, we give examples which show that dim(X) does not characterize the black-box complexity of ANN above the threshold.Our scheme for ANN in low-dimensional metric spaces is the first to yield efficient algorithms without relying on any additional assumptions on the input. In previous approaches (e.g., [K.L. Clarkson, Nearest neighbor queries in metric spaces, Discrete Comput. Geom. 22(1) (1999) 63-93; D. Karger, M. Ruhl, Finding nearest neighbors in growth-restricted metrics, in: 34th Annu. ACM Symp. on the Theory of Computing, 2002, pp. 63-66; R. Krauthgamer, J.R. Lee, Navigating nets: simple algorithms for proximity search, in: 15th Annu. ACM-SIAM Symp. on Discrete Algorithms, 2004, pp. 791-801; K. Hildrum, et al, A note on finding nearest neighbors in growth-restricted metrics, in: Proc. of the 15th Annu. ACM-SIAM Symp. on Discrete Algorithms, 2004, pp. 560-561]), even spaces with dim(X) = O(1) sometimes required Ω(n) query times. In view of the minimization of a nonsmooth nonconvex function f, we prove an abstract convergence result for descent methods satisfying a sufficient-decrease assumption, and allowing a relative error tolerance. Our result guarantees the convergence of bounded sequences, under the assumption that the function f satisfies the Kurdyka–Łojasiewicz inequality. This assumption allows to cover a wide range of problems, including nonsmooth semi-algebraic (or more generally tame) minimization. The specialization of our result to different kinds of structured problems provides several new convergence results for inexact versions of the gradient method, the proximal method, the forward–backward splitting algorithm, the gradient projection and some proximal regularization of the Gauss–Seidel method in a nonconvex setting. Our results are illustrated through feasibility problems, or iterative thresholding procedures for compressive sensing. We study an evolving spatial network in which sequentially arriving vertices are joined to existing vertices at random according to a rule that combines preference according to degree with preference according to spatial proximity. We investigate phase transitions in graph structure as the relative weighting of these two components of the attachment rule is varied. Previous work of one of the authors showed that when the geometric component is weak, the limiting degree sequence of the resulting graph coincides with that of the standard Barab 'asi--Albert preferential attachment model. We show that at the other extreme, in the case of a sufficiently strong geometric component, the limiting degree sequence coincides with that of a purely geometric model, the on-line nearest-neighbour graph, which is of interest in its own right and for which we prove some extensions of known results. We also show the presence of an intermediate regime, in which the behaviour differs significantly from both the on-line nearest-neighbour graph and the Barab 'asi--Albert model; in this regime, we obtain a stretched exponential upper bound on the degree sequence. Our results lend some mathematical support to simulation studies of Manna and Sen, while proving that the power law to stretched exponential phase transition occurs at a different point from the one conjectured by those authors. We focus on nonconvex and nonsmooth minimization problems with a composite objective, where the differentiable part of the objective is freed from the usual and restrictive global Lipschitz gradient continuity assumption. This longstanding smoothness restriction is pervasive in first order methods (FOM), and was recently circumvent for convex composite optimization by Bauschke, Bolte and Teboulle, through a simple and elegant framework which captures, all at once, the geometry of the function and of the feasible set. Building on this work, we tackle genuine nonconvex problems. We first complement and extend their approach to derive a full extended descent lemma by introducing the notion of smooth adaptable functions. We then consider a Bregman-based proximal gradient methods for the nonconvex composite model with smooth adaptable functions, which is proven to globally converge to a critical point under natural assumptions on the problem's data. To illustrate the power and potential of our general framework and results, we consider a broad class of quadratic inverse problems with sparsity constraints which arises in many fundamental applications, and we apply our approach to derive new globally convergent schemes for this class. This paper aims at building a variational approach to the dynamics of discrete topological singularities in two dimensions, based on Γ-convergence. We consider discrete systems, described by scalar functions defined on a square lattice and governed by periodic interaction potentials. Our main motivation comes from XY spin systems, described by the phase parameter, and screw dislocations, described by the displacement function. For these systems, we introduce a discrete notion of vorticity. As the lattice spacing tends to zero we derive the first order Γ-limit of the free energy which is referred to as renormalized energy and describes the interaction of vortices. As a byproduct of this analysis, we show that such systems exhibit increasingly many metastable configurations of singularities. Therefore, we propose a variational approach to the depinning and dynamics of discrete vortices, based on minimizing movements. We show that, letting first the lattice spacing and then the time step of the minimizing movements tend to zero, the vortices move according with the gradient flow of the renormalized energy, as in the continuous Ginzburg–Landau framework. We study a regularized version of Hastings-Levitov planar random growth that models clusters formed by the aggregation of diffusing particles. In this model, the growing clusters are defined in terms of iterated slit maps whose capacities are given by c_n=c| n-1 '(e^ +i )|^ - , 0, where c>0 is the capacity of the first particle, _n are the composed conformal maps defining the clusters of the evolution, _n are independent uniform angles determining the positions at which particles are attached, and >0 is a regularization parameter which we take to depend on c. We prove that under an appropriate rescaling of time, in the limit as c converges to 0, the clusters converge to growing disks with deterministic capacities, provided that does not converge to 0 too fast. We then establish scaling limits for the harmonic measure flow, showing that by letting tend to 0 at different rates it converges to either the Brownian web on the circle, a stopped version of the Brownian web on the circle, or the identity map. As the harmonic measure flow is closely related to the internal branching structure within the cluster, the above three cases intuitively correspond to the number of infinite branches in the model being either 1, a random number whose distribution we obtain, or unbounded, in the limit as c converges to 0. We also present several findings based on simulations of the model with parameter choices not covered by our rigorous analysis. We investigate a family of regression problems in a semi-supervised setting. The task is to assign real-valued labels to a set of @math sample points, provided a small training subset of @math labeled points. A goal of semi-supervised learning is to take advantage of the (geometric) structure provided by the large number of unlabeled data when assigning labels. We consider random geometric graphs, with connection radius @math , to represent the geometry of the data set. Functionals which model the task reward the regularity of the estimator function and impose or reward the agreement with the training data. Here we consider the discrete @math -Laplacian regularization. We investigate asymptotic behavior when the number of unlabeled points increases, while the number of training points remains fixed. We uncover a delicate interplay between the regularizing nature of the functionals considered and the nonlocality inherent to the graph constructions. We rigorously obtain almost optimal ranges on the scaling of @math for the asymptotic consistency to hold. We prove that the minimizers of the discrete functionals in random setting converge uniformly to the desired continuum limit. Furthermore we discover that for the standard model used there is a restrictive upper bound on how quickly @math must converge to zero as @math . We introduce a new model which is as simple as the original model, but overcomes this restriction. The full' edge isoperimetric inequality for the discrete cube (due to Harper, Bernstein, Lindsay and Hart) specifies the minimum size of the edge boundary @math of a set @math , as a function of @math . A weaker (but more widely-used) lower bound is @math , where equality holds iff @math is a subcube. In 2011, the first author obtained a sharp stability' version of the latter result, proving that if @math , then there exists a subcube @math such that @math . The weak' version of the edge isoperimetric inequality has the following well-known generalization for the @math -biased' measure @math on the discrete cube: if @math , or if @math and @math is monotone increasing, then @math . In this paper, we prove a sharp stability version of the latter result, which generalizes the aforementioned result of the first author. Namely, we prove that if @math , then there exists a subcube @math such that @math , where @math . This result is a central component in recent work of the authors proving sharp stability versions of a number of Erd o s-Ko-Rado type theorems in extremal combinatorics, including the seminal complete intersection theorem' of Ahlswede and Khachatrian. In addition, we prove a biased-measure analogue of the full' edge isoperimetric inequality, for monotone increasing sets, and we observe that such an analogue does not hold for arbitrary sets, hence answering a question of Kalai. We use this result to give a new proof of the full' edge isoperimetric inequality, one relying on the Kruskal-Katona theorem. We prove a conjecture raised by the work of Diaconis and Shahshahani (1981) about the mixing time of random walks on the permutation group induced by a given conjugacy class. To do this we exploit a connection with coalescence and fragmentation processes and control the Kantorovitch distance by using a variant of a coupling due to Oded Schramm. Recasting our proof in the language of Ricci curvature, our proof establishes the occurrence of a phase transition, which takes the following form in the case of random transpositions: at time @math , the curvature is asymptotically zero for @math and is strictly positive for @math . A particle in Rd moves in discrete time. The size of the nth step is of order 1 n and when the particle is at a position v the expectation of the next step is in the direction F(v) for some fixed vector function F of class C2. It is well known that the only possible points p where v(n) may converge are those satisfying F(p) = 0. This paper proves that convergence to some of these points is in fact impossible as long as the "noise" -the difference between each step and its expectation-is sufficiently omnidirectional. The points where convergence is impossible are the unstable critical points for the autonomous flow (d dt)v(t) = F(v(t)). This generalizes several known results that say convergence is impossible at a repelling node of the flow. The Ginzburg-Landau functional is a phase transition model which is suitable for clustering or classification type problems. We study the asymptotics of a sequence of Ginzburg-Landau functionals with anisotropic interaction potentials on point clouds @math where @math denotes the number data points. In particular we show the limiting problem, in the sense of @math -convergence, is related to the total variation norm restricted to functions taking binary values; which can be understood as a surface energy. We generalize the result known for isotropic interaction potentials to the anisotropic case and add a result concerning the rate of convergence. Author(s): Gennip, YV; Bertozzi, AL | Abstract: We study Γ-convergence of graph-based Ginzburg–Landau functionals, both the limit for zero diffusive interface parameter e → 0 and the limit for infinite nodes in the graph m → ∞. For general graphs we prove that in the limit e → 0 the graph cut objective function is recovered. We show that the continuum limit of this objective function on 4-regular graphs is related to the total variation seminorm and compare it with the limit of the discretized Ginzburg–Landau functional. For both functionals we also study the simultaneous limit e → 0 and m → ∞, by expressing e as a power of m and taking m → ∞. Finally we investigate the continuum limit for a nonlocal means-type functional on a completely connected graph This paper introduces a parallel and distributed algorithm for solving the following minimization problem with linear constraints: @math minimizef1(x1)+ź+fN(xN)subject toA1x1+ź+ANxN=c,x1źX1,ź,xNźXN,where @math Nź2, @math fi are convex functions, @math Ai are matrices, and @math Xi are feasible sets for variable @math xi. Our algorithm extends the alternating direction method of multipliers (ADMM) and decomposes the original problem into N smaller subproblems and solves them in parallel at each iteration. This paper shows that the classic ADMM can be extended to the N-block Jacobi fashion and preserve convergence in the following two cases: (i) matrices @math Ai are mutually near-orthogonal and have full column-rank, or (ii) proximal terms are added to the N subproblems (but without any assumption on matrices @math Ai). In the latter case, certain proximal terms can let the subproblem be solved in more flexible and efficient ways. We show that @math źxk+1-xkźM2 converges at a rate of o(1 k) where M is a symmetric positive semi-definte matrix. Since the parameters used in the convergence analysis are conservative, we introduce a strategy for automatically tuning the parameters to substantially accelerate our algorithm in practice. We implemented our algorithm (for the case ii above) on Amazon EC2 and tested it on basis pursuit problems with >300 GB of distributed data. This is the first time that successfully solving a compressive sensing problem of such a large scale is reported.