Maximization of non-submodular set functions has been attracting a lot of attention since many set functions that appear in practice lack submodularity. In this paper, we study maximization of weakly modular functions, which has both weak submodularity and weak supermodularity . Actually, weakly modular maximization (WMM) can model various important (possibly non-submodular) maximization problems including feature selection and production planning. We reveal two theoretical properties of WMM: We prove a hardness result related to improving an existing approximation guarantee achieved by the greedy algorithm, and we show that WMM is fixed-parameter tractable under certain conditions. The latter result provides a new time--accuracy trade-off for @math -constrained minimization as a corollary. We then prove that efficient multi-stage algorithms can perform well for WMM; the obtained theoretical guarantees generalize and improve some existing results. Experiments confirm the practical utility of the multi-stage algorithms.
proposed for atom selection, which basically corresponds to the case where @math is quadratic. However, its theoretical guarantee has not been proved. We extend their algorithm to the case where @math is RSC RSM and prove its guarantee. Many recent studies on @math -constrained minimization have been devoted to projected-gradient-based methods such as and @cite_32 @cite_27 @cite_23 , which are typically faster than greedy-style methods. However, their theoretical guarantees imply that projected-gradient-based methods can be affected by the ill condition more negatively than greedy-style methods. For example, to achieve @math -error guarantees, forward greedy selection and require @math and @math , respectively (see, @cite_21 @cite_32 ), where @math is a condition number ; a larger @math implies a worse condition.
@cite_27 @cite_21 @cite_32 @cite_23
2263717697 2156512439 2655474054 2318684976
We consider the problem of maximizing a (non-monotone) submodular function subject to a cardinality constraint. In addition to capturing well-known combinatorial optimization problems, e.g., Max-k-Coverage and Max-Bisection, this problem has applications in other more practical settings such as natural language processing, information retrieval, and machine learning. In this work we present improved approximations for two variants of the cardinality constraint for non-monotone functions. When at most k elements can be chosen, we improve the current best 1 e -- o(1) approximation to a factor that is in the range [1 e + 0.004, 1 2], achieving a tight approximation of 1 2 -- o(1) for k = n 2 and breaking the 1 e barrier for all values of k. When exactly k elements must be chosen, our algorithms improve the current best 1 4 -- o(1) approximation to a factor that is in the range [0.356, 1 2], again achieving a tight approximation of 1 2 -- o(1) for k = n 2. Additionally, some of the algorithms we provide are very fast with time complexities of O(nk), as opposed to previous known algorithms which are continuous in nature, and thus, too slow for applications in the practical settings mentioned above. Our algorithms are based on two new techniques. First, we present a simple randomized greedy approach where in each step a random element is chosen from a set of "reasonably good" elements. This approach might be considered a natural substitute for the greedy algorithm of Nemhauser, Wolsey and Fisher [45], as it retains the same tight guarantee of 1--1 e for monotone objectives and the same time complexity of O(nk), while giving an approximation of 1 e for general non-monotone objectives (while the greedy algorithm of Nemhauser et. al. fails to provide any constant guarantee). Second, we extend the double greedy technique, which achieves a tight 1 2 approximation for unconstrained submodular maximization, to the continuous setting. This allows us to manipulate the natural rates by which elements change, thus bounding the total number of elements chosen. In the first part of this paper we show a similarity between the principle of Structural Risk Minimization Principle (SRM) (Vapnik, 1982) and the idea of Sparse Approximation, as defined in (Chen, Donoho and Saunders, 1995) and Olshausen and Field (1996). Then we focus on two specific (approximate) implementations of SRM and Sparse Approximation, which have been used to solve the problem of function approximation. For SRM we consider the Support Vector Machine technique proposed by V. Vapnik and his team at AT &T Bell Labs, and for Sparse Approximation we consider a modification of the Basis Pursuit De-Noising algorithm proposed by Chen, Donoho and Saunders (1995). We show that, under certain conditions, these two techniques are equivalent: they give the same solution and they require the solution of the same quadratic programming problem. We focus on nonconvex and nonsmooth minimization problems with a composite objective, where the differentiable part of the objective is freed from the usual and restrictive global Lipschitz gradient continuity assumption. This longstanding smoothness restriction is pervasive in first order methods (FOM), and was recently circumvent for convex composite optimization by Bauschke, Bolte and Teboulle, through a simple and elegant framework which captures, all at once, the geometry of the function and of the feasible set. Building on this work, we tackle genuine nonconvex problems. We first complement and extend their approach to derive a full extended descent lemma by introducing the notion of smooth adaptable functions. We then consider a Bregman-based proximal gradient methods for the nonconvex composite model with smooth adaptable functions, which is proven to globally converge to a critical point under natural assumptions on the problem's data. To illustrate the power and potential of our general framework and results, we consider a broad class of quadratic inverse problems with sparsity constraints which arises in many fundamental applications, and we apply our approach to derive new globally convergent schemes for this class. In this article we present the first effective method based on global optimization for the reconstruction of image puzzles comprising rectangle pieces—Puzzle Solving by Quadratic Programming (PSQP). The proposed novel mathematical formulation reduces the problem to the maximization of a constrained quadratic function, which is solved via a gradient ascent approach. The proposed method is deterministic and can deal with arbitrary identical rectangular pieces. We provide experimental results showing its effectiveness when compared to state-of-the-art approaches. Although the method was developed to solve image puzzles, we also show how to apply it to the reconstruction of simulated strip-shredded documents, broadening its applicability.