We design new approximation algorithms for the problems of optimizing submodular and supermodular functions subject to a single matroid constraint. Specifically, we consider the case in which we wish to maximize a nondecreasing submodular function or minimize a nonincreasing supermodular function in the setting of bounded total curvature @math . In the case of submodular maximization with curvature @math , we obtain a @math -approximation --- the first improvement over the greedy @math -approximation of Conforti and Cornuejols from 1984, which holds for a cardinality constraint, as well as recent approaches that hold for an arbitrary matroid constraint. Our approach is based on modifications of the continuous greedy algorithm and non-oblivious local search, and allows us to approximately maximize the sum of a nonnegative, nondecreasing submodular function and a (possibly negative) linear function. We show how to reduce both submodular maximization and supermodular minimization to this general problem when the objective function has bounded total curvature. We prove that the approximation results we obtain are the best possible in the value oracle model, even in the case of a cardinality constraint. We define an extension of the notion of curvature to general monotone set functions and show @math -approximation for maximization and @math -approximation for minimization cases. Finally, we give two concrete applications of our results in the settings of maximum entropy sampling, and the column-subset selection problem.
Conforti and Cornu ' e jols @cite_18 defined the @math of non-decreasing submodular function @math as They showed that greedy algorithm has an approximation ratio of @math for the problem of maximizing a nondecreasing submodular function with curvature at most @math subject to a single matroid constraint. In the special case of a uniform matroid, they were able to show that the greedy is a @math -approximation algorithm. Later, Vondr ' a k @cite_15 considered the continuous greedy algorithm in the setting of bounded curvature. He introduced the notion of , which is a weaker notion than total curvature, and showed that the continuous greedy algorithm is a @math -approximation for maximizing a nondecreasing submodular function @math subject to an arbitrary matroid constraint whenever @math has curvature at most @math with respect to the optimum. He also showed that it is impossible to obtain a @math -approximation in this setting when evaluating @math on only a polynomial number of sets. Unfortunately, unlike total curvature, it is in general not possible to compute the curvature of a function with respect to the optimum, as it requires knowledge of an optimal solution.
@cite_18 @cite_15
2011193572 2310530842
Consider the problem of choosing a string of actions to optimize an objective function that is string submodular. It was shown in previous papers that the greedy strategy, consisting of a string of actions that only locally maximizes the step-wise gain in the objective function, achieves at least a @math -approximation to the optimal strategy. This paper improves this approximation by introducing additional constraints on curvature, namely, total backward curvature, total forward curvature, and elemental forward curvature. We show that if the objective function has total backward curvature @math , then the greedy strategy achieves at least a @math -approximation of the optimal strategy. If the objective function has total forward curvature @math , then the greedy strategy achieves at least a @math -approximation of the optimal strategy. Moreover, we consider a generalization of the diminishing-return property by defining the elemental forward curvature. We also introduce the notion of string-matroid and consider the problem of maximizing the objective function subject to a string-matroid constraint. We investigate two applications of string submodular functions with curvature constraints: 1) choosing a string of actions to maximize the expected fraction of accomplished tasks; and 2) designing a string of measurement matrices such that the information gain is maximized. The problem of choosing a string of actions to optimize an objective function that is string submodular has been considered in [1]. There it is shown that the greedy strategy, consisting of a string of actions that only locally maximizes the step-wise gain in the objective function, achieves at least a (1-e-1)-approximation to the optimal strategy. This paper improves this approximation by introducing additional constraints on curvature, namely, total backward curvature, total forward curvature, and elemental forward curvature. We show that if the objective function has total backward curvature σ, then the greedy strategy achieves at least a 1 σ(1 - e-σ)-approximation of the optimal strategy. If the objective function has total forward curvature c, then the greedy strategy achieves at least a (1 - e)-approximation of the optimal strategy. Moreover, we consider a generalization of the diminishing-return property by defining the elemental forward curvature. We also introduce the notion of string-matroid and consider the problem of maximizing the objective function subject to a string-matroid constraint. Let (X;I) be a matroid and let f : 2 X ! R+ be a monotone submodular function. The curvature of f is the minimum c 2 [0; 1] such that for any S X and j 2 X n S, f (S [ fjg) f (S) (1 c)f (fjg). We consider the optimization problem maxff (S) : S 2 Ig. It is known that the greedy algorithm yields a 1=2-approximation for this problem [10], and 1 1+c -approximation when f has curvature c [3]. For the uniform matroid, it was known that the greedy algorithm yields an improved 1 (1 e c )-approximation [3]. In this paper, we analyze the continuous greedy algorithm [19] and prove that it gives a 1 c (1 e c )-approximation for any matroid. Moreover, we show that this holds for a relaxed notion of curvature, curvature with respect to the optimum, and we prove that any better approximation under these conditions would require an exponential number of value queries. Let @math be a monotone submodular set function, and let @math be a matroid. We consider the problem @math . It is known that the greedy algorithm yields a @math -approximation [M. L. Fisher, G. L. Nemhauser, and L. A. Wolsey, Math. Programming Stud., no. 8 (1978), pp. 73-87] for this problem. For certain special cases, e.g., @math , the greedy algorithm yields a @math -approximation. It is known that this is optimal both in the value oracle model (where the only access to @math is through a black box returning @math for a given set @math ) [G. L. Nemhauser and L. A. Wolsey, Math. Oper. Res., 3 (1978), pp. 177-188] and for explicitly posed instances assuming @math [U. Feige, J. ACM, 45 (1998), pp. 634-652]. In this paper, we provide a randomized @math -approximation for any monotone submodular function and an arbitrary matroid. The algorithm works in the value oracle model. Our main tools are a variant of the pipage rounding technique of Ageev and Sviridenko [J. Combin. Optim., 8 (2004), pp. 307-328], and a continuous greedy process that may be of independent interest. As a special case, our algorithm implies an optimal approximation for the submodular welfare problem in the value oracle model [J. Vondrak, Proceedings of the @math th ACM Symposium on Theory of Computing, 2008, pp. 67-74]. As a second application, we show that the generalized assignment problem (GAP) is also a special case; although the reduction requires @math to be exponential in the original problem size, we are able to achieve a @math -approximation for GAP, simplifying previously known algorithms. Additionally, the reduction enables us to obtain approximation algorithms for variants of GAP with more general constraints. Abstract For the problem maxlcub; Z ( S ): S is an independent set in the matroid X rcub;, it is well-known that the greedy algorithm finds an optimal solution when Z is an additive set function (Rado-Edmonds theorem). Fisher, Nemhauser and Wolsey have shown that, when Z is a nondecreasing submodular set function satisfying Z (∅)=0, the greedy algorithm finds a solution with value at least half the optimum value. In this paper we show that it finds a solution with value at least 1 (1 + α) times the optimum value, where α is a parameter which represents the ‘total curvature’ of Z . This parameter satisfies 0≤α≤1 and α=0 if and only if the set function Z is additive. Thus the theorems of Rado-Edmonds and Fisher-Nemhauser-Wolsey are both contained in the bound 1 (1 + α). We show that this bound is best possible in terms of α. Another bound which generalizes the Rado-Edmonds theorem is given in terms of a ‘greedy curvature’ of the set function. Unlike the first bound, this bound can prove the optimality of the greedy algorithm even in instances where Z is not additive. A third bound, in terms of the rank and the girth of X , unifies and generalizes the bounds (e−1) e known for uniform matroids and 1 2 for general matroids. We also analyze the performance of the greedy algorithm when X is an independence system instead of a matroid. Then we derive two bounds, both tight: The first one is [1−(1−α K ) k ] α where K and k are the sizes of the largest and smallest maximal independent sets in X respectively; the second one is 1 ( p +α) where p is the minimum number of matroids that must be intersected to obtain X . For revenue and welfare maximization in single-dimensional Bayesian settings, (STOC10) recently showed that sequential posted-price mechanisms (SPMs), though simple in form, can perform surprisingly well compared to the optimal mechanisms. In this paper, we give a theoretical explanation of this fact, based on a connection to the notion of correlation gap. Loosely speaking, for auction environments with matroid constraints, we can relate the performance of a mechanism to the expectation of a monotone submodular function over a random set. This random set corresponds to the winner set for the optimal mechanism, which is highly correlated, and corresponds to certain demand set for SPMs, which is independent. The notion of correlation gap of (SODA10) quantifies how much we "lose" in the expectation of the function by ignoring correlation in the random set, and hence bounds our loss in using certain SPM instead of the optimal mechanism. Furthermore, the correlation gap of a monotone and submodular function is known to be small, and it follows that certain SPM can approximate the optimal mechanism by a good constant factor. Exploiting this connection, we give tight analysis of a greedy-based SPM of for several environments. In particular, we show that it gives an @math -approximation for matroid environments, gives asymptotically a @math -approximation for the important sub-case of @math -unit auctions, and gives a @math -approximation for environments with @math -independent set system constraints. In the Submodular Welfare Problem, m items are to be distributed among n players with utility functions wi: 2[m] → R+. The utility functions are assumed to be monotone and submodular. Assuming that player i receives a set of items Si, we wish to maximize the total utility ∑i=1n wi(Si). In this paper, we work in the value oracle model where the only access to the utility functions is through a black box returning wi(S) for a given set S. Submodular Welfare is in fact a special case of the more general problem of submodular maximization subject to a matroid constraint: max f(S): S ∈ I , where f is monotone submodular and I is the collection of independent sets in some matroid. For both problems, a greedy algorithm is known to yield a 1 2-approximation [21, 16]. In special cases where the matroid is uniform (I = S: |S| ≤ k) [20] or the submodular function is of a special type [4, 2], a (1-1 e)-approximation has been achieved and this is optimal for these problems in the value oracle model [22, 6, 15]. A (1-1 e)-approximation for the general Submodular Welfare Problem has been known only in a stronger demand oracle model [4], where in fact 1-1 e can be improved [9]. In this paper, we develop a randomized continuous greedy algorithm which achieves a (1-1 e)-approximation for the Submodular Welfare Problem in the value oracle model. We also show that the special case of n equal players is approximation resistant, in the sense that the optimal (1-1 e)-approximation is achieved by a uniformly random solution. Using the pipage rounding technique [1, 2], we obtain a (1-1 e)-approximation for submodular maximization subject to any matroid constraint. The continuous greedy algorithm has a potential of wider applicability, which we demonstrate on the examples of the Generalized Assignment Problem and the AdWords Assignment Problem. We consider the problem of maximizing a nonnegative submodular set function @math over a ground set @math subject to a variety of packing-type constraints including (multiple) matroid constraints, knapsack constraints, and their intersections. In this paper we develop a general framework that allows us to derive a number of new results, in particular, when @math may be a nonmonotone function. Our algorithms are based on (approximately) maximizing the multilinear extension @math of @math over a polytope @math that represents the constraints, and then effectively rounding the fractional solution. Although this approach has been used quite successfully, it has been limited in some important ways. We overcome these limitations as follows. First, we give constant factor approximation algorithms to maximize @math over a downward-closed polytope @math described by an efficient separation oracle. Previously this was known only for monotone functions. For nonmonotone functions, a constant factor was ... We consider the problem of optimization from samples of monotone submodular functions with bounded curvature. In numerous applications, the function optimized is not known a priori, but instead learned from data. What are the guarantees we have when optimizing functions from sampled data? In this paper we show that for any monotone submodular function with curvature c there is a (1 - c) (1 + c - c^2) approximation algorithm for maximization under cardinality constraints when polynomially-many samples are drawn from the uniform distribution over feasible sets. Moreover, we show that this algorithm is optimal. That is, for any c < 1, there exists a submodular function with curvature c for which no algorithm can achieve a better approximation. The curvature assumption is crucial as for general monotone submodular functions no algorithm can obtain a constant-factor approximation for maximization under a cardinality constraint when observing polynomially-many samples drawn from any distribution over feasible sets, even when the function is statistically learnable. The study of combinatorial problems with a sub modular objective function has attracted much attention in recent years, and is partly motivated by the importance of such problems to economics, algorithmic game theory and combinatorial optimization. Classical works on these problems are mostly combinatorial in nature. Recently, however, many results based on continuous algorithmic tools have emerged. The main bottleneck of such continuous techniques is how to approximately solve a non-convex relaxation for the sub modular problem at hand. Thus, the efficient computation of better fractional solutions immediately implies improved approximations for numerous applications. A simple and elegant method, called continuous greedy'', successfully tackles this issue for monotone sub modular objective functions, however, only much more complex tools are known to work for general non-monotone sub modular objectives. In this work we present a new unified continuous greedy algorithm which finds approximate fractional solutions for both the non-monotone and monotone cases, and improves on the approximation ratio for many applications. For general non-monotone sub modular objective functions, our algorithm achieves an improved approximation ratio of about @math . For monotone sub modular objective functions, our algorithm achieves an approximation ratio that depends on the density of the poly tope defined by the problem at hand, which is always at least as good as the previously known best approximation ratio of @math . Some notable immediate implications are an improved @math -approximation for maximizing a non-monotone sub modular function subject to a matroid or @math -knapsack constraints, and information-theoretic tight approximations for Sub modular Max-SAT and Sub modular Welfare with @math players, for any number of players @math . A framework for sub modular optimization problems, called the , was introduced recently by The improved approximation ratio of the unified continuous greedy algorithm implies improved approximation ratios for many problems through this framework. Moreover, via a parameter called , our algorithm merges the relaxation solving and re-normalization steps of the framework, and achieves, for some applications, further improvements. We also describe new monotone balanced contention resolution schemes for various matching, scheduling and packing problems, thus, improving the approximations achieved for these problems via the framework. Submodular function maximization is a central problem in combinatorial optimization, generalizing many important NP-hard problems including max cut in digraphs, graphs, and hypergraphs; certain constraint satisfaction problems; maximum entropy sampling; and maximum facility location problems. Our main result is that for any k ≥ 2 and any e > 0, there is a natural local search algorithm that has approximation guarantee of 1 (k + e) for the problem of maximizing a monotone submodular function subject to k matroid constraints. This improves upon the 1 (k + 1)-approximation of Fisher, Nemhauser, and Wolsey obtained in 1978 [Fisher, M., G. Nemhauser, L. Wolsey. 1978. An analysis of approximations for maximizing submodular set functions---II. Math. Programming Stud.8 73--87]. Also, our analysis can be applied to the problem of maximizing a linear objective function and even a general nonmonotone submodular function subject to k matroid constraints. We show that, in these cases, the approximation guarantees of our algorithms are 1 (k-1 + e) and 1 (k + 1 + 1 (k-1) + e), respectively. Our analyses are based on two new exchange properties for matroids. One is a generalization of the classical Rota exchange property for matroid bases, and another is an exchange property for two matroids based on the structure of matroid intersection. Is it possible to maximize a monotone submodular function faster than the widely used lazy greedy algorithm (also known as accelerated greedy), both in theory and practice? In this paper, we develop the first linear-time algorithm for maximizing a general monotone submodular function subject to a cardinality constraint. We show that our randomized algorithm, STOCHASTIC-GREEDY, can achieve a @math approximation guarantee, in expectation, to the optimum solution in time linear in the size of the data and independent of the cardinality constraint. We empirically demonstrate the effectiveness of our algorithm on submodular functions arising in data summarization, including training large-scale kernel methods, exemplar-based clustering, and sensor placement. We observe that STOCHASTIC-GREEDY practically achieves the same utility value as lazy greedy but runs much faster. More surprisingly, we observe that in many practical scenarios STOCHASTIC-GREEDY does not evaluate the whole fraction of data points even once and still achieves indistinguishable results compared to lazy greedy. Constrained submodular maximization problems have long been studied, most recently in the context of auctions and computational advertising, with near-optimal results known under a variety of constraints when the submodular function is monotone. In this paper, we give constant approximation algorithms for the non-monotone case that work for p-independence systems (which generalize constraints given by the intersection of p matroids that had been studied previously), where the running time is poly(n, p). Our algorithms and analyses are simple, and essentially reduce non-monotone maximization to multiple runs of the greedy algorithm previously used in the monotone case. We extend these ideas to give a simple greedy-based constant factor algorithms for non-monotone submodular maximization subject to a knapsack constraint, and for (online) secretary setting (where elements arrive one at a time in random order and the algorithm must make irrevocable decisions) subject to uniform matroid or a partition matroid constraint. Finally, we give an O(log k) approximation in the secretary setting subject to a general matroid constraint of rank k. This paper presents a polynomial-time @math -approximation algorithm for maximizing nonnegative @math -submodular functions. This improves upon the previous @math -approximation by Ward and Z ivn 'y (SODA'14), where @math . We also show that for monotone @math -submodular functions there is a polynomial-time @math -approximation algorithm while for any @math a @math -approximation algorithm for maximizing monotone @math -submodular functions would require exponentially many queries. In particular, our hardness result implies that our algorithms are asymptotically tight. We also extend the approach to provide constant factor approximation algorithms for maximizing skew-bisubmodular functions, which were recently introduced as generalizations of bisubmodular functions. We consider the problem of maximizing a nonnegative (possibly non-monotone) submodular set function with or without constraints. [9] showed a 2 5-approximation for the unconstrained problem and also proved that no approximation better than 1 2 is possible in the value oracle model. Constant-factor approximation has been also known for submodular maximization subject to a matroid independence constraint (a factor of 0.309 [33]) and for submodular maximization subject to a matroid base constraint, provided that the fractional base packing number ν is bounded away from 1 (a 1 4-approximation assuming that ν ≥ 2 [33]). In this paper, we propose a new algorithm for submodular maximization which is based on the idea of simulated annealing. We prove that this algorithm achieves improved approximation for two problems: a 0.41-approximation for unconstrained submodular maximization, and a 0.325-approximation for submodular maximization subject to a matroid independence constraint. On the hardness side, we show that in the value oracle model it is impossible to achieve a 0.478-approximation for submodular maximization subject to a matroid independence constraint, or a 0.394-approximation subject to a matroid base constraint in matroids with two disjoint bases. Even for the special case of cardinality constraint, we prove it is impossible to achieve a 0.491-approximation. (Previously it was conceivable that a 1 2-approximation exists for these problems.) It is still an open question whether a 1 2-approximation is possible for unconstrained submodular maximization. In this paper, we study the problem of maximizing continuous submodular functions that naturally arise in many learning applications such as those involving utility functions in active learning and sensing, matrix approximations and network inference. Despite the apparent lack of convexity in such functions, we prove that stochastic projected gradient methods can provide strong approximation guarantees for maximizing continuous submodular functions with convex constraints. More specifically, we prove that for monotone continuous DR-submodular functions, all fixed points of projected gradient ascent provide a factor @math approximation to the global maxima. We also study stochastic gradient methods and show that after @math iterations these methods reach solutions which achieve in expectation objective values exceeding @math . An immediate application of our results is to maximize submodular functions that are defined stochastically, i.e. the submodular function is defined as an expectation over a family of submodular functions with an unknown distribution. We will show how stochastic gradient methods are naturally well-suited for this setting, leading to a factor @math approximation when the function is monotone. In particular, it allows us to approximately maximize discrete, monotone submodular optimization problems via projected gradient ascent on a continuous relaxation, directly connecting the discrete and continuous domains. Finally, experiments on real data demonstrate that our projected gradient methods consistently achieve the best utility compared to other continuous baselines while remaining competitive in terms of computational effort. We investigate the approximability of several classes of real-valued functions by functions of a small number of variables ( juntas ). Our main results are tight bounds on the number of variables required to approximate a function @math within @math -error @math over the uniform distribution: 1. If @math is submodular, then it is @math -close to a function of @math variables. This is an exponential improvement over previously known results. We note that @math variables are necessary even for linear functions. 2. If @math is fractionally subadditive (XOS) it is @math -close to a function of @math variables. This result holds for all functions with low total @math -influence and is a real-valued analogue of Friedgut's theorem for boolean functions. We show that @math variables are necessary even for XOS functions. As applications of these results, we provide learning algorithms over the uniform distribution. For XOS functions, we give a PAC learning algorithm that runs in time @math . For submodular functions we give an algorithm in the more demanding PMAC learning model (Balcan and Harvey, 2011) which requires a multiplicative @math factor approximation with probability at least @math over the target distribution. Our uniform distribution algorithm runs in time @math . This is the first algorithm in the PMAC model that over the uniform distribution can achieve a constant approximation factor arbitrarily close to 1 for all submodular functions. As follows from the lower bounds in (, 2013) both of these algorithms are close to optimal. We also give applications for proper learning, testing and agnostic learning with value queries of these classes. Abstract We introduce and study rough (approximate) lower curvature bounds for discrete spaces and for graphs. This notion agrees with the one introduced in [J. Lott, C. Villani, Ricci curvature for metric-measure spaces via optimal transport, Ann. of Math. 169 (2009), in press] and [K.T. Sturm, On the geometry of metric measure spaces. I, Acta Math. 196 (2006) 65–131], in the sense that the metric measure space which is approximated by a sequence of discrete spaces with rough curvature ⩾ K will have curvature ⩾ K in the sense of [J. Lott, C. Villani, Ricci curvature for metric-measure spaces via optimal transport, Ann. of Math. 169 (2009), in press; K.T. Sturm, On the geometry of metric measure spaces. I, Acta Math. 196 (2006) 65–131]. Moreover, in the converse direction, discretizations of metric measure spaces with curvature ⩾ K will have rough curvature ⩾ K . We apply our results to concrete examples of homogeneous planar graphs. We consider the problem of maximizing a monotone submodular function subject to a knapsack constraint. Our main contribution is an algorithm that achieves a nearly-optimal, @math approximation, using @math function evaluations and arithmetic operations. Our algorithm is impractical but theoretically interesting, since it overcomes a fundamental running time bottleneck of the multilinear extension relaxation framework. This is the main approach for obtaining nearly-optimal approximation guarantees for important classes of constraints but it leads to @math running times, since evaluating the multilinear extension is expensive. Our algorithm maintains a fractional solution with only a constant number of entries that are strictly fractional, which allows us to overcome this obstacle. This paper proposes low-complexity algorithms for finding approximate second-order stationary points (SOSPs) of problems with smooth non-convex objective and linear constraints. While finding (approximate) SOSPs is computationally intractable, we first show that generic instances of the problem can be solved efficiently. More specifically, for a generic problem instance, certain strict complementarity (SC) condition holds for all Karush-Kuhn-Tucker (KKT) solutions (with probability one). The SC condition is then used to establish an equivalence relationship between two different notions of SOSPs, one of which is computationally easy to verify. Based on this particular notion of SOSP, we design an algorithm named the Successive Negative-curvature grAdient Projection (SNAP), which successively performs either conventional gradient projection or some negative curvature based projection steps to find SOSPs. SNAP and its first-order extension SNAP @math , require @math iterations to compute an @math -SOSP, and their per-iteration computational complexities are polynomial in the number of constraints and problem dimension. To our knowledge, this is the first time that first-order algorithms with polynomial per-iteration complexity and global sublinear rate have been designed to find SOSPs of the important class of non-convex problems with linear constraints.