Submodular functions can be exactly minimized in polynomial time, and the special case that graph cuts solve with max flow KZ:PAMI04 has had significant impact in computer vision BVZ:PAMI01,Kwatra:SIGGRAPH03,Rother:GrabCut04 . In this paper we address the important class of sum-of-submodular (SoS) functions Arora:ECCV12,Kolmogorov:DAM12 , which can be efficiently minimized via a variant of max flow called submodular flow Edmonds:ADM77 . SoS functions can naturally express higher order priors involving, e.g., local image patches; however, it is difficult to fully exploit their expressive power because they have so many parameters. Rather than trying to formulate existing higher order priors as an SoS function, we take a discriminative learning approach, effectively searching the space of SoS functions for a higher order prior that performs well on our training set. We adopt a structural SVM approach Joachims et al 09a,Tsochantaridis et al 04 and formulate the training problem in terms of quadratic programming; as a result we can efficiently search the space of SoS priors via an extended cutting-plane algorithm. We also show how the state-of-the-art max flow method for vision problems Goldberg:ESA11 can be modified to efficiently solve the submodular flow problem. Experimental comparisons are made against the OpenCV implementation of the GrabCut interactive segmentation technique Rother:GrabCut04 , which uses hand-tuned parameters instead of machine learning. On a standard dataset Gulshan:CVPR10 our method learns higher order priors with hundreds of parameter values, and produces significantly better segmentations. While our focus is on binary labeling problems, we show that our techniques can be naturally generalized to handle more than two labels.
Learning of submodular discriminant functions where a prediction is computed through maximization has widespread use in information retrieval, where submodularity models diversity in the ranking of a search engine @cite_5 @cite_15 or in an automatically generated abstract @cite_19 . While exact (monotone) submodular maximization is intractible, approximate inference using a simple greedy algorithm has approximation guarantees and generally excellent performance in practice.
@cite_19 @cite_5 @cite_15
1771852678 1570200465 52746146 2545520746
We consider the maximization problem in the value oracle model of functions defined on k-tuples of sets that are submodular in every orthant and r-wise monotone, where k g 2 and 1 l r l k. We give an analysis of a deterministic greedy algorithm that shows that any such function can be approximated to a factor of 1 (1 + r). For r e k, we give an analysis of a randomized greedy algorithm that shows that any such function can be approximated to a factor of 1 (1+sk 2. In the case of k e r e 2, the considered functions correspond precisely to bisubmodular functions, in which case we obtain an approximation guarantee of 1 2. We show that, as in the case of submodular functions, this result is the best possible both in the value query model and under the assumption that NP ≠ RP. Extending a result of , we show that for any k g 3, submodularity in every orthant and pairwise monotonicity (i.e., r e 2) precisely characterize k-submodular functions. Consequently, we obtain an approximation guarantee of 1 3 (and thus independent of k) for the maximization problem of k-submodular functions. Many large-scale machine learning problems--clustering, non-parametric learning, kernel machines, etc.--require selecting a small yet representative subset from a large dataset. Such problems can often be reduced to maximizing a submodular set function subject to various constraints. Classical approaches to submodular optimization require centralized access to the full dataset, which is impractical for truly large-scale problems. In this paper, we consider the problem of submodular function maximization in a distributed fashion. We develop a simple, two-stage protocol GreeDi, that is easily implemented using MapReduce style computations. We theoretically analyze our approach, and show that under certain natural conditions, performance close to the centralized approach can be achieved. We begin with monotone submodular maximization subject to a cardinality constraint, and then extend this approach to obtain approximation guarantees for (not necessarily monotone) submodular maximization subject to more general constraints including matroid or knapsack constraints. In our extensive experiments, we demonstrate the effectiveness of our approach on several applications, including sparse Gaussian process inference and exemplar based clustering on tens of millions of examples using Hadoop. Motivated by extremely large-scale machine learning problems, we introduce a new multi-stage algorithmic framework for submodular maximization (called MULTGREED), where at each stage we apply an approximate greedy procedure to maximize surrogate submodular functions. The surrogates serve as proxies for a target sub-modular function but require less memory and are easy to evaluate. We theoretically analyze the performance guarantee of the multi-stage framework and give examples on how to design instances of MULTGREED for a broad range of natural submodular functions. We show that MULTGREED performs very closely to the standard greedy algorithm given appropriate surrogate functions and argue how our framework can easily be integrated with distributive algorithms for further optimization. We complement our theory by empirically evaluating on several real-world problems, including data subset selection on millions of speech samples where MULTGREED yields at least a thousand times speedup and superior results over the state-of-the-art selection methods. Submodularity is one of the most important properties of combinatorial optimization, and k-submodularity is a generalization of submodularity. Maximization of a k-submodular function is NP-hard, and approximation algorithms have been studied. Most of algorithms use randomization and achieve the approximation ratio as the expected value. For unconstrained submodular maximization, [Buchbinder and Feldman 2016] gave a derandomization scheme, and showed that randomness is not necessary for obtaining optimal results. In this paper, we extend their scheme for unconstrained k-submodular maximization and give two deterministic algorithms for k-submodular maximization. The first is for monotone k-submodular functions. We derandomize (k (2k-1) )-approximation algorithm [Iwata, Tanigawa, and Yoshida 2016], and achieve the same approximation ratio. The second is for nonmonotone functions with (k 2 ). It achieves (k (3k-2) )-approximation, and it is better than a simple greedy algorithm [Ward and Živný 2016] which achieves 1 3-approximation for (k 3 ).