We design new approximation algorithms for the problems of optimizing submodular and supermodular functions subject to a single matroid constraint. Specifically, we consider the case in which we wish to maximize a nondecreasing submodular function or minimize a nonincreasing supermodular function in the setting of bounded total curvature @math . In the case of submodular maximization with curvature @math , we obtain a @math -approximation --- the first improvement over the greedy @math -approximation of Conforti and Cornuejols from 1984, which holds for a cardinality constraint, as well as recent approaches that hold for an arbitrary matroid constraint. Our approach is based on modifications of the continuous greedy algorithm and non-oblivious local search, and allows us to approximately maximize the sum of a nonnegative, nondecreasing submodular function and a (possibly negative) linear function. We show how to reduce both submodular maximization and supermodular minimization to this general problem when the objective function has bounded total curvature. We prove that the approximation results we obtain are the best possible in the value oracle model, even in the case of a cardinality constraint. We define an extension of the notion of curvature to general monotone set functions and show @math -approximation for maximization and @math -approximation for minimization cases. Finally, we give two concrete applications of our results in the settings of maximum entropy sampling, and the column-subset selection problem.
We shall also consider the problem of minimizing nonincreasing functions @math . By analogy with total curvature, Il'ev @cite_25 defines the @math of a nonincreasing supermodular function. His definition, which is stated in terms of the marginal of the function, is equivalent to when reformulated in terms of marginal gains. He showed that, in contrast to submodular maximization, the simple greedy heuristic does not give a constant factor approximation algorithm in the general case. However, when the supermodular function @math has total curvature at most @math , he shows that the reverse greedy algorithm is an @math -approximation where @math .
@cite_25
2030375943
Consider the problem of choosing a string of actions to optimize an objective function that is string submodular. It was shown in previous papers that the greedy strategy, consisting of a string of actions that only locally maximizes the step-wise gain in the objective function, achieves at least a @math -approximation to the optimal strategy. This paper improves this approximation by introducing additional constraints on curvature, namely, total backward curvature, total forward curvature, and elemental forward curvature. We show that if the objective function has total backward curvature @math , then the greedy strategy achieves at least a @math -approximation of the optimal strategy. If the objective function has total forward curvature @math , then the greedy strategy achieves at least a @math -approximation of the optimal strategy. Moreover, we consider a generalization of the diminishing-return property by defining the elemental forward curvature. We also introduce the notion of string-matroid and consider the problem of maximizing the objective function subject to a string-matroid constraint. We investigate two applications of string submodular functions with curvature constraints: 1) choosing a string of actions to maximize the expected fraction of accomplished tasks; and 2) designing a string of measurement matrices such that the information gain is maximized. The problem of choosing a string of actions to optimize an objective function that is string submodular has been considered in [1]. There it is shown that the greedy strategy, consisting of a string of actions that only locally maximizes the step-wise gain in the objective function, achieves at least a (1-e-1)-approximation to the optimal strategy. This paper improves this approximation by introducing additional constraints on curvature, namely, total backward curvature, total forward curvature, and elemental forward curvature. We show that if the objective function has total backward curvature σ, then the greedy strategy achieves at least a 1 σ(1 - e-σ)-approximation of the optimal strategy. If the objective function has total forward curvature c, then the greedy strategy achieves at least a (1 - e)-approximation of the optimal strategy. Moreover, we consider a generalization of the diminishing-return property by defining the elemental forward curvature. We also introduce the notion of string-matroid and consider the problem of maximizing the objective function subject to a string-matroid constraint. This paper considers stochastic optimization problems whose objective functions involve powers of random variables. For example, consider the classic Stochastic lp Load Balancing Problem (SLBp): There are @math machines and @math jobs, and known independent random variables @math decribe the load incurred on machine @math if we assign job @math to it. The goal is to assign each jobs to machines in order to minimize the expected @math -norm of the total load on the machines. While convex relaxations represent one of the most powerful algorithmic tools, in problems such as SLBp the main difficulty is to capture the objective function in a way that only depends on each random variable separately. We show how to capture @math -power-type objectives in such separable way by using the @math -function method, introduced by Lata a to relate the moment of sums of random variables to the individual marginals. We show how this quickly leads to a constant-factor approximation for very general subset selection problem with @math -moment objective. Moreover, we give a constant-factor approximation for SLBp, improving on the recent @math -approximation of [, SODA 18]. Here the application of the method is much more involved. In particular, we need to sharply connect the expected @math -norm of a random vector with the @math -moments of its marginals (machine loads), taking into account simultaneously the different scales of the loads that are incurred by an unknown assignment. This paper presents a polynomial-time @math -approximation algorithm for maximizing nonnegative @math -submodular functions. This improves upon the previous @math -approximation by Ward and Z ivn 'y (SODA'14), where @math . We also show that for monotone @math -submodular functions there is a polynomial-time @math -approximation algorithm while for any @math a @math -approximation algorithm for maximizing monotone @math -submodular functions would require exponentially many queries. In particular, our hardness result implies that our algorithms are asymptotically tight. We also extend the approach to provide constant factor approximation algorithms for maximizing skew-bisubmodular functions, which were recently introduced as generalizations of bisubmodular functions. We investigate the approximability of several classes of real-valued functions by functions of a small number of variables ( juntas ). Our main results are tight bounds on the number of variables required to approximate a function @math within @math -error @math over the uniform distribution: 1. If @math is submodular, then it is @math -close to a function of @math variables. This is an exponential improvement over previously known results. We note that @math variables are necessary even for linear functions. 2. If @math is fractionally subadditive (XOS) it is @math -close to a function of @math variables. This result holds for all functions with low total @math -influence and is a real-valued analogue of Friedgut's theorem for boolean functions. We show that @math variables are necessary even for XOS functions. As applications of these results, we provide learning algorithms over the uniform distribution. For XOS functions, we give a PAC learning algorithm that runs in time @math . For submodular functions we give an algorithm in the more demanding PMAC learning model (Balcan and Harvey, 2011) which requires a multiplicative @math factor approximation with probability at least @math over the target distribution. Our uniform distribution algorithm runs in time @math . This is the first algorithm in the PMAC model that over the uniform distribution can achieve a constant approximation factor arbitrarily close to 1 for all submodular functions. As follows from the lower bounds in (, 2013) both of these algorithms are close to optimal. We also give applications for proper learning, testing and agnostic learning with value queries of these classes. We consider the problem of maximizing a nonnegative submodular set function @math over a ground set @math subject to a variety of packing-type constraints including (multiple) matroid constraints, knapsack constraints, and their intersections. In this paper we develop a general framework that allows us to derive a number of new results, in particular, when @math may be a nonmonotone function. Our algorithms are based on (approximately) maximizing the multilinear extension @math of @math over a polytope @math that represents the constraints, and then effectively rounding the fractional solution. Although this approach has been used quite successfully, it has been limited in some important ways. We overcome these limitations as follows. First, we give constant factor approximation algorithms to maximize @math over a downward-closed polytope @math described by an efficient separation oracle. Previously this was known only for monotone functions. For nonmonotone functions, a constant factor was ... Let @math be a monotone submodular set function, and let @math be a matroid. We consider the problem @math . It is known that the greedy algorithm yields a @math -approximation [M. L. Fisher, G. L. Nemhauser, and L. A. Wolsey, Math. Programming Stud., no. 8 (1978), pp. 73-87] for this problem. For certain special cases, e.g., @math , the greedy algorithm yields a @math -approximation. It is known that this is optimal both in the value oracle model (where the only access to @math is through a black box returning @math for a given set @math ) [G. L. Nemhauser and L. A. Wolsey, Math. Oper. Res., 3 (1978), pp. 177-188] and for explicitly posed instances assuming @math [U. Feige, J. ACM, 45 (1998), pp. 634-652]. In this paper, we provide a randomized @math -approximation for any monotone submodular function and an arbitrary matroid. The algorithm works in the value oracle model. Our main tools are a variant of the pipage rounding technique of Ageev and Sviridenko [J. Combin. Optim., 8 (2004), pp. 307-328], and a continuous greedy process that may be of independent interest. As a special case, our algorithm implies an optimal approximation for the submodular welfare problem in the value oracle model [J. Vondrak, Proceedings of the @math th ACM Symposium on Theory of Computing, 2008, pp. 67-74]. As a second application, we show that the generalized assignment problem (GAP) is also a special case; although the reduction requires @math to be exponential in the original problem size, we are able to achieve a @math -approximation for GAP, simplifying previously known algorithms. Additionally, the reduction enables us to obtain approximation algorithms for variants of GAP with more general constraints. For revenue and welfare maximization in single-dimensional Bayesian settings, (STOC10) recently showed that sequential posted-price mechanisms (SPMs), though simple in form, can perform surprisingly well compared to the optimal mechanisms. In this paper, we give a theoretical explanation of this fact, based on a connection to the notion of correlation gap. Loosely speaking, for auction environments with matroid constraints, we can relate the performance of a mechanism to the expectation of a monotone submodular function over a random set. This random set corresponds to the winner set for the optimal mechanism, which is highly correlated, and corresponds to certain demand set for SPMs, which is independent. The notion of correlation gap of (SODA10) quantifies how much we "lose" in the expectation of the function by ignoring correlation in the random set, and hence bounds our loss in using certain SPM instead of the optimal mechanism. Furthermore, the correlation gap of a monotone and submodular function is known to be small, and it follows that certain SPM can approximate the optimal mechanism by a good constant factor. Exploiting this connection, we give tight analysis of a greedy-based SPM of for several environments. In particular, we show that it gives an @math -approximation for matroid environments, gives asymptotically a @math -approximation for the important sub-case of @math -unit auctions, and gives a @math -approximation for environments with @math -independent set system constraints. Convex and submodular functions play an important role in many applications, and in particular in combinatorial optimization. Here we study two special cases: convexity in one dimension and submodularity in two dimensions. The latter type of functions are equivalent to the well-known Monge matrices. A matrix @math is called a Monge matrix if for every @math and @math we have @math . If inequality holds in the opposite direction, then V is an inverse Monge matrix (supermodular function). Many problems, such as the traveling salesperson problem and various transportation problems, can be solved more efficiently if the input is a Monge matrix. In this work we present testing algorithms for the above properties. A testing algorithm for a predetermined property @math is given query access to an unknown function f and a distance parameter @math . The algorithm should accept f with high probability if it has the property @math and reject it with high probability if more than an @math -fraction of the function values should be modified so that f obtains the property. Our algorithm for testing whether a 1-dimensional function @math is convex (concave) has query complexity and running time of @math . Our algorithm for testing whether an n1 × n2 matrix V is a Monge (inverse Monge) matrix has query complexity and running time of @math . In this paper, we study the problem of maximizing continuous submodular functions that naturally arise in many learning applications such as those involving utility functions in active learning and sensing, matrix approximations and network inference. Despite the apparent lack of convexity in such functions, we prove that stochastic projected gradient methods can provide strong approximation guarantees for maximizing continuous submodular functions with convex constraints. More specifically, we prove that for monotone continuous DR-submodular functions, all fixed points of projected gradient ascent provide a factor @math approximation to the global maxima. We also study stochastic gradient methods and show that after @math iterations these methods reach solutions which achieve in expectation objective values exceeding @math . An immediate application of our results is to maximize submodular functions that are defined stochastically, i.e. the submodular function is defined as an expectation over a family of submodular functions with an unknown distribution. We will show how stochastic gradient methods are naturally well-suited for this setting, leading to a factor @math approximation when the function is monotone. In particular, it allows us to approximately maximize discrete, monotone submodular optimization problems via projected gradient ascent on a continuous relaxation, directly connecting the discrete and continuous domains. Finally, experiments on real data demonstrate that our projected gradient methods consistently achieve the best utility compared to other continuous baselines while remaining competitive in terms of computational effort. We consider the Unconstrained Submodular Maximization problem in which we are given a nonnegative submodular function @math , and the objective is to find a subset @math maximizing @math . This is one of the most basic submodular optimization problems, having a wide range of applications. Some well-known problems captured by Unconstrained Submodular Maximization include Max-Cut, Max-DiCut, and variants of Max-SAT and maximum facility location. We present a simple randomized linear time algorithm achieving a tight approximation guarantee of 1 2, thus matching the known hardness result of Feige, Mirrokni, and Vondrak [SIAM J. Comput., 40 (2011), pp. 1133--1153]. Our algorithm is based on an adaptation of the greedy approach which exploits certain symmetry properties of the problem. We consider the problem of minimizing a supermodular set function whose special case is the well-known NP-hard p-median problem. The main result of the paper is a tight bound on the approximation ratio of a greedy heuristic (discrete analog of the steepest descent algorithm) for this problem. Submodular optimization has found many applications in machine learning and beyond. We carry out the first systematic investigation of inference in probabilistic models defined through submodular functions, generalizing regular pairwise MRFs and Determinantal Point Processes. In particular, we present L-FIELD, a variational approach to general log-submodular and log-supermodular distributions based on sub- and supergradients. We obtain both lower and upper bounds on the log-partition function, which enables us to compute probability intervals for marginals, conditionals and marginal likelihoods. We also obtain fully factorized approximate posteriors, at the same computational cost as ordinary submodular optimization. Our framework results in convex problems for optimizing over differentials of submodular functions, which we show how to optimally solve. We provide theoretical guarantees of the approximation quality with respect to the curvature of the function. We further establish natural relations between our variational approach and the classical mean-field method. Lastly, we empirically demonstrate the accuracy of our inference scheme on several submodular models. We consider the following generalization of the binary search problem. A search strategy is required to locate an unknown target node @math in a given tree @math . Upon querying a node @math of the tree, the strategy receives as a reply an indication of the connected component of @math containing the target @math . The cost of querying each node is given by a known non-negative weight function, and the considered objective is to minimize the total query cost for a worst-case choice of the target. Designing an optimal strategy for a weighted tree search instance is known to be strongly NP-hard, in contrast to the unweighted variant of the problem which can be solved optimally in linear time. Here, we show that weighted tree search admits a quasi-polynomial time approximation scheme: for any @math , there exists a @math -approximation strategy with a computation time of @math . Thus, the problem is not APX-hard, unless @math . By applying a generic reduction, we obtain as a corollary that the studied problem admits a polynomial-time @math -approximation. This improves previous @math -approximation approaches, where the @math -notation disregards @math -factors. A Fixed-Parameter Tractable ( ) @math -approximation algorithm for a minimization (resp. maximization) parameterized problem @math is an FPT algorithm that, given an instance @math computes a solution of cost at most @math (resp. @math ) if a solution of cost at most (resp. at least) @math exists; otherwise the output can be arbitrary. For well-known intractable problems such as the W[1]-hard Clique and W[2]-hard Set Cover problems, the natural question is whether we can get any -approximation. It is widely believed that both Clique and Set-Cover admit no FPT @math -approximation algorithm, for any increasing function @math . Assuming standard conjectures such as the Exponential Time Hypothesis (ETH) eth-paturi and the Projection Games Conjecture (PGC) r3 , we make the first progress towards proving this conjecture by showing that 1. Under the ETH and PGC, there exist constants @math such that the Set Cover problem does not admit an FPT approximation algorithm with ratio @math in @math time, where @math is the size of the universe and @math is the number of sets. 2. Unless @math , for every @math there exists a constant @math such that Clique has no FPT cost approximation with ratio @math in @math time, where @math is the number of vertices in the graph. In the second part of the paper we consider various W[1]-hard problems such as , , Directed Steiner Network and . For all these problem we give polynomial time @math -approximation algorithms for some small function @math (the largest approximation ratio we give is @math ). We consider the problem of estimating the unknown response function in the multichannel deconvolution model with a boxcar-like kernel which is of particular interest in signal processing. It is known that, when the number of channels is finite, the precision of reconstruction of the response function increases as the number of channels @math grow (even when the total number of observations @math for all channels @math remains constant) and this requires that the parameter of the channels form a Badly Approximable @math -tuple. Recent advances in data collection and recording techniques made it of urgent interest to study the case when the number of channels @math grow with the total number of observations @math . However, in real-life situations, the number of channels @math usually refers to the number of physical devices and, consequently, may grow to infinity only at a slow rate as @math . When @math grows slowly as @math increases, we develop a procedure for the construction of a Badly Approximable @math -tuple on a specified interval, of a non-asymptotic length, together with a lower bound associated with this @math -tuple, which explicitly shows its dependence on @math as @math is growing. This result is further used for the evaluation of the @math -risk of the suggested adaptive wavelet thresholding estimator of the unknown response function and, furthermore, for the choice of the optimal number of channels @math which minimizes the @math -risk. Consider a convex relaxation @math of a pseudo-boolean function @math . We say that the relaxation is totally half-integral if @math is a polyhedral function with half-integral extreme points @math , and this property is preserved after adding an arbitrary combination of constraints of the form @math , @math , and @math where @math is a constant. A well-known example is the roof duality relaxation for quadratic pseudo-boolean functions @math . We argue that total half-integrality is a natural requirement for generalizations of roof duality to arbitrary pseudo-boolean functions. Our contributions are as follows. First, we provide a complete characterization of totally half-integral relaxations @math by establishing a one-to-one correspondence with bisubmodular functions . Second, we give a new characterization of bisubmodular functions. Finally, we show some relationships between general totally half-integral relaxations and relaxations based on the roof duality. We consider @math -median clustering in finite metric spaces and @math -means clustering in Euclidean spaces, in the setting where @math is part of the input (not a constant). For the @math -means problem, show that if the optimal @math -means clustering of the input is more expensive than the optimal @math -means clustering by a factor of @math , then one can achieve a @math -approximation to the @math -means optimal in time polynomial in @math and @math by using a variant of Lloyd's algorithm. In this work we substantially improve this approximation guarantee. We show that given only the condition that the @math -means optimal is more expensive than the @math -means optimal by a factor @math for some constant @math , we can obtain a PTAS. In particular, under this assumption, for any @math we achieve a @math -approximation to the @math -means optimal in time polynomial in @math and @math , and exponential in @math and @math . We thus decouple the strength of the assumption from the quality of the approximation ratio. We also give a PTAS for the @math -median problem in finite metrics under the analogous assumption as well. For @math -means, we in addition give a randomized algorithm with improved running time of @math . Our technique also obtains a PTAS under the assumption of that all @math approximations are @math -close to a desired target clustering, in the case that all target clusters have size greater than @math and @math is constant. Note that the motivation of is that for many clustering problems, the objective function is only a proxy for the true goal of getting close to the target. From this perspective, our improvement is that for @math -means in Euclidean spaces we reduce the distance of the clustering found to the target from @math to @math when all target clusters are large, and for @math -median we improve the largeness'' condition needed in the work of to get exactly @math -close from @math to @math . Our results are based on a new notion of clustering stability. We study the problem of fair allocation for indivisible goods. We use the the maxmin share paradigm introduced by Budish as a measure for fairness. Procacciafirst (EC'14) were first to investigate this fundamental problem in the additive setting. In contrast to what real-world experiments suggest, they show that a maxmin guarantee (1- @math allocation) is not always possible even when the number of agents is limited to 3. While the existence of an approximation solution (e.g. a @math - @math allocation) is quite straightforward, improving the guarantee becomes subtler for larger constants. Procaccia provide a proof for existence of a @math - @math allocation and leave the question open for better guarantees. Our main contribution is an answer to the above question. We improve the result of ! to a @math factor in the additive setting. The main idea for our @math - @math allocation method is clustering the agents. To this end, we introduce three notions and techniques, namely reducibility, matching allocation, and cycle-envy-freeness, and prove the approximation guarantee of our algorithm via non-trivial applications of these techniques. Our analysis involves coloring and double counting arguments that might be of independent interest. One major shortcoming of the current studies on fair allocation is the additivity assumption on the valuations. We alleviate this by extending our results to the case of submodular, fractionally subadditive, and subadditive settings. More precisely, we give constant approximation guarantees for submodular and XOS agents, and a logarithmic approximation for the case of subadditive agents. Furthermore, we complement our results by providing close upper bounds for each class of valuation functions. Finally, we present algorithms to find such allocations for additive, submodular, and XOS settings in polynomial time. We give the first dimension-efficient algorithms for learning Rectified Linear Units (ReLUs), which are functions of the form @math with @math . Our algorithm works in the challenging Reliable Agnostic learning model of Kalai, Kanade, and Mansour (2009) where the learner is given access to a distribution @math on labeled examples but the labeling may be arbitrary. We construct a hypothesis that simultaneously minimizes the false-positive rate and the loss on inputs given positive labels by @math , for any convex, bounded, and Lipschitz loss function. The algorithm runs in polynomial-time (in @math ) with respect to any distribution on @math (the unit sphere in @math dimensions) and for any error parameter @math (this yields a PTAS for a question raised by F. Bach on the complexity of maximizing ReLUs). These results are in contrast to known efficient algorithms for reliably learning linear threshold functions, where @math must be @math and strong assumptions are required on the marginal distribution. We can compose our results to obtain the first set of efficient algorithms for learning constant-depth networks of ReLUs. Our techniques combine kernel methods and polynomial approximations with a "dual-loss" approach to convex programming. As a byproduct we obtain a number of applications including the first set of efficient algorithms for "convex piecewise-linear fitting" and the first efficient algorithms for noisy polynomial reconstruction of low-weight polynomials on the unit sphere.