In Valiant's model of evolution, a class of representations is evolvable iff a polynomial-time process of random mutations guided by selection converges with high probability to a representation as @math -close as desired from the optimal one, for any required @math . Several previous positive results exist that can be related to evolving a vector space, but each former result imposes restrictions either on (re)initialisations, distributions, performance functions and or the mutator. In this paper, we show that all it takes to evolve a complete normed vector space is merely a set that generates the space. Furthermore, it takes only @math steps and it is essentially strictly monotonic, agnostic and handles target drifts that rival some proven in fairly restricted settings. In the context of the model, we bring to the fore new results not documented previously. Evolution appears to occur in a mean-divergence model reminiscent of Markowitz mean-variance model for portfolio selection, and the risk-return efficient frontier of evolution shows an interesting pattern: when far from the optimum, the mutator always has access to mutations close to the efficient frontier. Toy experiments in supervised and unsupervised learning display promising directions for this scheme to be used as a (new) provable gradient-free stochastic optimisation algorithm.
Our main result suffers none of these downsides: our mutator meets time and space optimality properties (Section ), we do not change the set of mutations (Section ), we do not do restart. Also, our evolvability scheme is agnostic (Section ), stable (Section ) and handles significant drift (Section ). Finally, instead of fixed-degree polynomials, we consider any finite-valued function @math . Thus, we can evolve functions with infinite Taylor expansion, something @cite_1 does not cover It is also not clear whether a simple trick to extend @cite_1 --- replacing variables by bounded functions --- is possible without endangering the distribution support assumption or the complexity parameters. . Finally, our mutator yields an extremely simple and provable evolutionary scheme, implementable using few lines of code as sketched in Algorithm ( ).
@cite_1
2109000177 1892716203 48624319 2015766439
Extending previous analyses on function classes like linear functions, we analyze how the simple 1+1 evolutionary algorithm optimizes pseudo-Boolean functions that are strictly monotonic. These functions have the property that whenever only 0-bits are changed to 1, then the objective value strictly increases. Contrary to what one would expect, not all of these functions are easy to optimize. The choice of the constant c in the mutation probability pn=c n can make a decisive difference. We show that if c iterations. For c=1, we can still prove an upper bound of On3 2. However, for , we present a strictly monotonic function such that the 1+1 EA with overwhelming probability needs iterations to find the optimum. This is the first time that we observe that a constant factor change of the mutation probability changes the runtime by more than a constant factor. Extending previous analyses on function classes like linear functions, we analyze how the simple (1+1) evolutionary algorithm optimizes pseudo-Boolean functions that are strictly monotone. Contrary to what one would expect, not all of these functions are easy to optimize. The choice of the constant c in the mutation probability p(n) = c n can make a decisive difference. We show that if c > 1, then the (1+1) EA finds the optimum of every such function in Θ(n log n) iterations. For c = 1, we can still prove an upper bound of O(n3 2). However, for c > 33, we present a strictly monotone function such that the (1+1) EA with overwhelming probability does not find the optimum within 2Ω(n) iterations. This is the first time that we observe that a constant factor change of the mutation probability changes the run-time by more than constant factors. Motivated by the problem of querying and communicating bidders' valuations in combinatorial auctions, we study how well different classes of set functions can be sketched. More formally, let f be a function mapping subsets of some ground set [n] to the non-negative real numbers. We say that f' is an α-sketch of f if for every set S, the value f'(S) lies between f(S) α and f(S), and f' can be specified by poly(n) bits. We show that for every subadditive function f there exists an α-sketch where α = n1 2. O(polylog(n)). Furthermore, we provide an algorithm that finds these sketches with a polynomial number of demand queries. This is essentially the best we can hope for since: 1. We show that there exist subadditive functions (in fact, XOS functions) that do not admit an o(n1 2) sketch. (Balcan and Harvey [3] previously showed that there exist functions belonging to the class of substitutes valuations that do not admit an O(n1 3) sketch.) 2. We prove that every deterministic algorithm that accesses the function via value queries only cannot guarantee a sketching ratio better than n1−e. We also show that coverage functions, an interesting subclass of submodular functions, admit arbitrarily good sketches. Finally, we show an interesting connection between sketching and learning. We show that for every class of valuations, if the class admits an α-sketch, then it can be α-approximately learned in the PMAC model of Balcan and Harvey. The bounds we prove are only information-theoretic and do not imply the existence of computationally efficient learning algorithms in general. Abstract We analyze how the ( 1 + λ ) evolutionary algorithm (EA) optimizes linear pseudo-Boolean functions. We prove that it finds the optimum of any linear function within an expected number of O ( 1 λ n log ⁡ n + n ) iterations. We also show that this bound is sharp for some linear functions, e.g., the binary value function. Since previous works shows an asymptotically smaller runtime for the special case of OneMax , it follows that for the ( 1 + λ ) EA different linear functions may have run-times of different asymptotic order. The proof of our upper bound heavily relies on a number of classic and recent drift analysis methods. In particular, we show how to analyze a process displaying different types of drifts in different phases. Our work corrects a wrongfully claimed better asymptotic runtime in an earlier work [13] . We also use our methods to analyze the runtime of the ( 1 + λ ) EA on the OneMax test function and obtain a new upper bound of O ( n log ⁡ log ⁡ λ log ⁡ λ ) for the case that λ is larger than O ( log ⁡ n log ⁡ log ⁡ n log ⁡ log ⁡ log ⁡ n ) ; this is the cut-off point where a linear speed-up ceases to exist. While our results are mostly spurred from a theory-driven interest, they also show that choosing the right size of the offspring population can be crucial. For both the binary value and the OneMax test function we observe that once a linear speed-up ceases to exist, in fact, the speed-up from a larger λ reduces to sub-logarithmic (still at the price of a linear increase of the cost of each generation).