Minimisation of discrete energies defined over factors is an important problem in computer vision, and a vast number of MAP inference algorithms have been proposed. Different inference algorithms perform better on factor graph models (GMs) from different underlying problem classes, and in general it is difficult to know which algorithm will yield the lowest energy for a given GM. To mitigate this difficulty, survey papers advise the practitioner on what algorithms perform well on what classes of models. We take the next step forward, and present a technique to automatically select the best inference algorithm for an input GM. We validate our method experimentally on an extended version of the OpenGM2 benchmark, containing a diverse set of vision problems. On average, our method selects an inference algorithm yielding labellings with 96 of variables the same as the best available algorithm.
Automatic algorithm selection was pioneered by @cite_5 , which considered algorithms for quadrature and process scheduling. More recently, machine learning techniques have been used to select algorithms for constraint-satisfaction @cite_49 , and other combinatorial search problems @cite_42 . However, none of these works consider selecting MAP inference algorithms.
@cite_5 @cite_42 @cite_49
1495775210 1483287332 2147148915
Publisher Summary The problem of selecting an effective algorithm arises in a wide variety of situations. This chapter starts with a discussion on abstract models: the basic model and associated problems, the model with selection based on features, and the model with variable performance criteria. One objective of this chapter is to explore the applicability of the approximation theory to the algorithm selection problem. There is an intimate relationship here and that the approximation theory forms an appropriate base upon which to develop a theory of algorithm selection methods. The approximation theory currently lacks much of the necessary machinery for the algorithm selection problem. There is a need to develop new results and apply known techniques to these new circumstances. The final pages of this chapter form a sort of appendix, which lists 15 specific open problems and questions in this area. There is a close relationship between the algorithm selection problem and the general optimization theory. This is not surprising since the approximation problem is a special form of the optimization problem. Most realistic algorithm selection problems are of moderate to high dimensionality and thus one should expect them to be quite complex. One consequence of this is that most straightforward approaches (even well-conceived ones) are likely to lead to enormous computations for the best selection. The single most important part of the solution of a selection problem is the appropriate choice of the form for selection mapping. It is here that theories give the least guidance and that the art of problem solving is most crucial. Machine learning is an established method of selecting algorithms to solve hard search problems. Despite this, to date no systematic comparison and evaluation of the different techniques has been performed and the performance of existing systems has not been critically compared to other approaches. We compare machine learning techniques for algorithm selection on real-world data sets of hard search problems. In addition to well-established approaches, for the first time we also apply statistical relational learning to this problem. We demonstrate that most machine learning techniques and existing systems perform less well than one might expect. To guide practitioners, we close by giving clear recommendations as to which machine learning techniques are likely to perform well based on our experiments. It has been widely observed that there is no single "dominant" SAT solver; instead, different solvers perform best on different instances. Rather than following the traditional approach of choosing the best solver for a given class of instances, we advocate making this decision online on a per-instance basis. Building on previous work, we describe SATzilla, an automated approach for constructing per-instance algorithm portfolios for SAT that use so-called empirical hardness models to choose among their constituent solvers. This approach takes as input a distribution of problem instances and a set of component solvers, and constructs a portfolio optimizing a given objective function (such as mean runtime, percent of instances solved, or score in a competition). The excellent performance of SATzilla was independently verified in the 2007 SAT Competition, where our SATzilla07 solvers won three gold, one silver and one bronze medal. In this article, we go well beyond SATzilla07 by making the portfolio construction scalable and completely automated, and improving it by integrating local search solvers as candidate solvers, by predicting performance score instead of runtime, and by using hierarchical hardness models that take into account different types of SAT instances. We demonstrate the effectiveness of these new techniques in extensive experimental results on data sets including instances from the most recent SAT competition.