The OneMax problem is a standard benchmark optimisation problem for a binary search space. Recent work on applying a Bandit-Based Random Mutation Hill-Climbing algorithm to the noisy OneMax Problem showed that it is important to choose a good value for the resampling number to make a careful trade off between taking more samples in order to reduce noise, and taking fewer samples to reduce the total computational cost. This paper extends that observation, by deriving an analytical expression for the running time of the RMHC algorithm with resampling applied to the noisy OneMax problem, and showing both theoretically and empirically that the optimal resampling number increases with the number of dimensions in the search space.
Droste @cite_6 defined a model for the OneMax problem, in which during every fitness evaluation of the bit-string @math , there was exactly one uniformly-chosen random bit mis-read with probability @math . Hence the measured noisy fitness values are equivalent to replacing the Gaussian noise term in by an appropriate discrete random variable taking values from @math . Under this scheme, Droste showed that (1+1)-EA with mutation probability @math could optimise a @math -bit OneMax problem corrupted by , with high probability, in polynomial time if @math is @math .
@cite_6
1569002781
We study the well-known black-box optimisation algorithm (1+1)-EA on a novel type of noise model. In our noise model, the fitness function is linear with positive weights, but the absolute values of the weights may fluctuate in each round. Thus in every state, the fitness function indicates that one-bits are preferred over zero-bits. In particular, hillclimbing heuristics should be able to find the optimum fast. We show that the (1+1)-EA indeed finds the optimum in time @math if the mutation parameter is @math for a constant @math However, we also show that for @math the (1+1)-EA needs superpolynomial time to find the optimum. Thus the choice of mutation parameter is critical even for optimisation tasks in which there is a clear path to the the optimum. A similar threshold phenomenon has recently been shown for noise-free monotone fitness functions. The ( @math 1+ź) EA with mutation probability c n, where @math c>0 is an arbitrary constant, is studied for the classical OneMax function. Its expected optimization time is analyzed exactly (up to lower order terms) as a function of c and @math ź. It turns out that 1 n is the only optimal mutation probability if @math ź=o(lnnlnlnn lnlnlnn), which is the cut-off point for linear speed-up. However, if @math ź is above this cut-off point then the standard mutation probability 1 n is no longer the only optimal choice. Instead, the expected number of generations is (up to lower order terms) independent of c, irrespectively of it being less than 1 or greater. The theoretical results are obtained by a careful study of order statistics of the binomial distribution and variable drift theorems for upper and lower bounds. Experimental supplements shed light on the optimal mutation probability for small problem sizes. We prove the first nontrivial (superlinear) lower bound in the noisy broadcast model, defined by El Gamal in [Open problems presented at the @math workshop on Specific Problems in Communication and Computation sponsored by Bell Communication Research, in Open Problems in Communication and Computation, T. M. Cover and B. Gopinath, eds., Springer-Verlag, New York, 1987, pp. 60-62]. In this model there are @math processors @math , each of which is initially given a private input bit @math . The goal is for @math to learn the value of @math , for some specified function @math , using a series of noisy broadcasts. At each step a designated processor broadcasts one bit to all of the other processors, and the bit received by each processor is flipped with fixed probability (independently for each recipient). In 1988, Gallager [IEEE Trans. Inform. Theory, 34 (1988), pp. 176-180] gave a noise-resistant protocol that allows @math to learn the entire input with constant probability in @math broadcasts. We prove that Gallager's protocol is optimal, up to a constant factor. Our lower bound follows by reduction from a lower bound for generalized noisy decision trees, a new model which may be of independent interest. For this new model we show a lower bound of @math on the depth of a tree that learns the entire input. While the above lower bound is for an @math -bit function, we also show an @math lower bound for the number of broadcasts required to compute certain explicit boolean-valued functions, when the correct output must be attained with probability at least @math for a constant parameter @math (this bound applies to all threshold functions as well as any other boolean-valued function with linear sensitivity). This bound also follows by reduction from a lower bound of @math on the depth of generalized noisy decision trees that compute the same functions with the same error. We also show a (nontrivial) @math lower bound on the depth of generalized noisy decision trees that compute such functions with small constant error. Finally, we show the first protocol in the noisy broadcast model that computes the Hamming weight of the input using a linear number of broadcasts. We solve the dynamic Predecessor Problem with high probability (whp) in constant time, using only @math bits of memory, for any constant @math . The input keys are random wrt a wider class of the well studied and practically important class of @math -smooth distributions introduced in and:mat . It achieves O(1) whp amortized time. Its worst-case time is @math . Also, we prove whp @math time using only @math bits. Finally, we show whp @math time using O(n) space. In practical applications evaluating a fitness function is frequently subject to noise, i. e., the “true fitness” is disturbed by some random variations. Evolutionary algorithms (EAs) are often successfully applied to noisy problems, where they have turned out to be particularly robust. Theoretical results on the behavior of EAs for noisy functions are comparatively very rare, especially for discrete search spaces. Here we present an analysis of the (1 + 1) EA for a noisy variant of OneMax and compute the maximal noise strength allowing the (1 + 1) EA a polynomial runtime asymptotically exactly. The methods used in the proofs are presented in a general form with clearly stated conditions in order to simplify further applications. Let @math denote a random symmetric @math matrix whose upper diagonal entries are independent and identically distributed Bernoulli random variables (which take values @math and @math with probability @math each). It is widely conjectured that @math is singular with probability at most @math . On the other hand, the best known upper bound on the singularity probability of @math , due to Vershynin (2011), is @math , for some unspecified small constant @math . This improves on a polynomial singularity bound due to Costello, Tao, and Vu (2005), and a bound of Nguyen (2011) showing that the singularity probability decays faster than any polynomial. In this paper, improving on all previous results, we show that the probability of singularity of @math is at most @math for all sufficiently large @math . The proof utilizes and extends a novel combinatorial approach to discrete random matrix theory, which has been recently introduced by the authors together with Luh and Samotij. Goldreich (ECCC 2000) suggested a simple construction of a candidate one-way function f : 0, 1 n → 0, 1 m where each bit of output is a fixed predicate P of a constant number d of (random) input bits. We investigate the security of this construction in the regime m = Dn, where D(d) is a sufficiently large constant. We prove that for any predicate P that correlates with either one or two of its inputs, f can be inverted with high probability. We also prove an amplification claim regarding Goldreich’s construction. Suppose we are given an assignment @math that has correlation @math with the hidden assignment @math . Then, given access to x′, it is possible to invert f on x with high probability, provided @math is sufficiently large. Consider the recovery of an unknown signal @math from quantized linear measurements. In the one-bit compressive sensing setting, one typically assumes that @math is sparse, and that the measurements are of the form @math . Since such measurements give no information on the norm of @math , recovery methods typically assume that @math . We show that if one allows more generally for quantized affine measurements of the form @math , and if the vectors @math are random, an appropriate choice of the affine shifts @math allows norm recovery to be easily incorporated into existing methods for one-bit compressive sensing. In addition, we show that for arbitrary fixed @math in the annulus @math , one may estimate the norm @math up to additive error @math from @math such binary measurements through a single evaluation of the inverse Gaussian error function. Finally, all of our recovery guarantees can be made universal over sparse vectors in the sense that with high probability, one set of measurements and thresholds can successfully estimate all sparse vectors @math in a Euclidean ball of known radius. Consider the recovery of an unknown signal @math from quantized linear measurements. In the one-bit compressive sensing setting, one typically assumes that @math is sparse, and that the measurements are of the form @math . Since such measurements give no information on the norm of @math , recovery methods from such measurements typically assume that @math . We show that if one allows more generally for quantized affine measurements of the form @math , and if the vectors @math are random, an appropriate choice of the affine shifts @math allows norm recovery to be easily incorporated into existing methods for one-bit compressive sensing. Additionally, we show that for arbitrary fixed @math in the annulus @math , one may estimate the norm @math up to additive error @math from @math such binary measurements through a single evaluation of the inverse Gaussian error function. Finally, all of our recovery guarantees can be made universal over sparse vectors, in the sense that with high probability, one set of measurements and thresholds can successfully estimate all sparse vectors @math within a Euclidean ball of known radius. Motivated by applications in blockchains and sensor networks, we consider a model of @math nodes trying to reach consensus on their majority bit. Each node @math is assigned a bit at time zero, and is a finite automaton with @math bits of memory (i.e., @math states) and a Poisson clock. When the clock of @math rings, @math can choose to communicate, and is then matched to a uniformly chosen node @math . The nodes @math and @math may update their states based on the state of the other node. Previous work has focused on minimizing the time to consensus and the probability of error, while our goal is minimizing the number of communications. We show that when @math , consensus can be reached at linear communication cost, but this is impossible if @math . We also study a synchronous variant of the model, where our upper and lower bounds on @math for achieving linear communication cost are @math and @math , respectively. A key step is to distinguish when nodes can become aware of knowing the majority bit and stop communicating. We show that this is impossible if their memory is too low. Given data stream @math of size @math of numbers from @math , the frequency of @math is defined as @math . The @math -th of @math is defined as @math . We consider the problem of approximating frequency moments in insertion-only streams for @math . For any constant @math we show an @math upper bound on the space complexity of the problem. Here @math is the iterative @math function. To simplify the presentation, we make the following assumptions: @math and @math are polynomially far; approximation error @math and parameter @math are constants. We observe a natural bijection between streams and special matrices. Our main technical contribution is a non-uniform sampling method on matrices. We call our method a ; it samples a heavy element (i.e., element @math with frequency @math ) with probability @math and gives approximation @math . In addition, the estimations never exceed the real values, that is @math for all @math . As a result, we reduce the space complexity of finding a heavy element to @math bits. We apply our method of recursive sketches and resolve the problem with @math bits. We present a randomised distributed maximal independent set (MIS) algorithm for arbitrary graphs of size @math that halts in time @math with probability @math , each message containing @math bit: thus its bit complexity per channel is @math (the bit complexity is the number of bits we need to solve a distributed task, it measures the communication complexity). We assume that the graph is anonymous: unique identities are not available to distinguish the processes; we only assume that each vertex distinguishes between its neighbours by locally known channel names. Furthermore we do not assume that the size (or an upper bound on the size) of the graph is known. This algorithm is optimal (modulo a multiplicative constant) for the bit complexity and improves the best previous randomised distributed MIS algorithms (deduced from the randomised PRAM algorithm due to Luby) for general graphs which is @math per channel (it halts in time @math and the size of each message is @math ). This result is based on a powerful and general technique for converting unrealistic exchanges of messages containing real numbers drawn at random on each vertex of a network into exchanges of bits. Then we consider a natural question: what is the impact of a vertex inclusion in the MIS on distant vertices? We prove that this impact vanishes rapidly as the distance grows for bounded-degree vertices. We provide a counter-example that shows this result does not hold in general. We prove also that these results remain valid for Luby's algorithm presented by Lynch and by Wattenhofer. This question remains open for the variant given by Peleg. Let @math be a uniformly distributed random @math -SAT formula with @math variables and @math clauses. We present a polynomial time algorithm that finds a satisfying assignment of @math with high probability for constraint densities @math , where @math . Previously no efficient algorithm was known to find satisfying assignments with a nonvanishing probability beyond @math [A. Frieze and S. Suen, J. Algorithms, 20 (1996), pp. 312-355]. The @math -fold direct sum encoding of a string @math is a function @math that takes as input sets @math of size @math and outputs @math . In this paper we prove a direct sum testing theorem. We describe a three query test that accepts with probability one any function of the form @math for some @math and rejects with probability @math functions @math that are @math -far from being a direct sum encoding, where the constant behind the @math notation is independent of @math . This theorem has a couple of additional guises: Linearity testing: By identifying the subsets of @math with vectors in @math in the natural way, our result can be thought of as a linearity testing theorem for functions whose domain is restricted to the @math th layer of the hypercube (i.e., the set of @math -bit strings with Hamming weight @math ). Tensor power testing: By moving to @math notation, the direct sum encoding is equivalent (up to a difference thatis negligi... In this paper, we consider a synchronization problem between nodes @math and @math that are connected through a two--way communication channel. Node @math contains a binary file @math of length @math and node @math contains a binary file @math that is generated by randomly deleting bits from @math , by a small deletion rate @math . The location of deleted bits is not known to either node @math or node @math . We offer a deterministic synchronization scheme between nodes @math and @math that needs @math transmissions and reconstructs @math at node @math with probability of error that is exponentially low in the size of @math . The rate of our scheme matches the optimal rate for this channel. Our scheme can be extended to other editing models, e.g., insertions, repetitions or replacements, as long as the rate of editing is small. We present an algorithm for computing @math , the @math th moment of an @math -dimensional frequency vector of a data stream, for @math , to within @math factors, @math with high constant probability. Let @math be the number of stream records and @math be the largest magnitude of a stream update. The algorithm uses space in bits @math where, @math . Here @math is @math for @math and @math for @math . This improves upon the space required by current algorithms iw:stoc05,bgks:soda06,ako:arxiv10,bo:arxiv10 by a factor of at least @math . The update time is @math . We use a new technique for designing estimators for functions of the form @math , where, @math is a random variable and @math is a smooth function, based on a low-degree Taylor polynomial expansion of @math around an estimate of @math . An algorithm of the form @math , where @math is a smooth function on @math , @math is a sequence of @math -valued random variables, @math is a sequence of independent standard d-dimensional Gaussian random variables, @math and @math for k large, is considered. An algorithm of this type arises by adding slowly decreasing white Gaussian noise to a stochastic gradient algorithm. It is shown, under suitable conditions on @math , @math , A, and B, that @math converges in probability to the set of global minima of @math . No prior information is assumed as to what bounded region contains a global minimum. The analysis is based on the asymptotic behavior of the related diffusion process @math , where @math is a standard d-dimensional Wiener process and @math for t large. Random constraint satisfaction problems (CSPs) are known to exhibit threshold phenomena: given a uniformly random instance of a CSP with @math variables and @math clauses, there is a value of @math beyond which the CSP will be unsatisfiable with high probability. Strong refutation is the problem of certifying that no variable assignment satisfies more than a constant fraction of clauses; this is the natural algorithmic problem in the unsatisfiable regime (when @math ). Intuitively, strong refutation should become easier as the clause density @math grows, because the contradictions introduced by the random clauses become more locally apparent. For CSPs such as @math -SAT and @math -XOR, there is a long-standing gap between the clause density at which efficient strong refutation algorithms are known, @math , and the clause density at which instances become unsatisfiable with high probability, @math . In this paper, we give spectral and sum-of-squares algorithms for strongly refuting random @math -XOR instances with clause density @math in time @math or in @math rounds of the sum-of-squares hierarchy, for any @math and any integer @math . Our algorithms provide a smooth transition between the clause density at which polynomial-time algorithms are known at @math , and brute-force refutation at the satisfiability threshold when @math . We also leverage our @math -XOR results to obtain strong refutation algorithms for SAT (or any other Boolean CSP) at similar clause densities. Our algorithms match the known sum-of-squares lower bounds due to Grigoriev and Schonebeck, up to logarithmic factors. Additionally, we extend our techniques to give new results for certifying upper bounds on the injective tensor norm of random tensors. Recently a great deal of attention has been focused on quantum computation following a sequence of results [Bernstein and Vazirani, in Proc. 25th Annual ACM Symposium Theory Comput., 1993, pp. 11--20, SIAM J. Comput., 26 (1997), pp. 1277--1339], [Simon, in Proc. 35th Annual IEEE Symposium Foundations Comput. Sci., 1994, pp. 116--123, SIAM J. Comput., 26 (1997), pp. 1340--1349], [Shor, in Proc. 35th Annual IEEE Symposium Foundations Comput. Sci., 1994, pp. 124--134] suggesting that quantum computers are more powerful than classical probabilistic computers. Following Shor's result that factoring and the extraction of discrete logarithms are both solvable in quantum polynomial time, it is natural to ask whether all of @math can be efficiently solved in quantum polynomial time. In this paper, we address this question by proving that relative to an oracle chosen uniformly at random with probability 1 the class @math cannot be solved on a quantum Turing machine (QTM) in time @math . We also show that relative to a permutation oracle chosen uniformly at random with probability 1 the class @math cannot be solved on a QTM in time @math . The former bound is tight since recent work of Grover [in Proc. @math th Annual ACM Symposium Theory Comput. , 1996] shows how to accept the class @math relative to any oracle on a quantum computer in time @math . The problem of estimating the phases (angles) of a complex unit-modulus vector @math from their noisy pairwise relative measurements @math , where @math is a complex-valued Gaussian random matrix, is known as phase synchronization. The maximum likelihood estimator (MLE) is a solution to a unit--modulus-constrained quadratic programming problem, which is nonconvex. Existing works have proposed polynomial-time algorithms such as a semidefinite programming (SDP) relaxation or the generalized power method (GPM). Numerical experiments suggest that both of these methods succeed with high probability for @math up to @math , yet existing analyses only confirm this observation for @math up to @math . In this paper, we bridge the gap by proving that the SDP relaxation is tight for @math , and GPM converges to the global optimum under the same regime. Moreover, we establish a linear convergence rate for GPM, and derive a tight...