In a steady-state evolution, tournament selection traditionally uses the fitness function to select the parents, and negative selection chooses an individual to be replaced with an offspring. This contribution focuses on analyzing the behavior, in terms of performance, of different heuristics when used instead of the fitness function in tournament selection. The heuristics analyzed are related to measuring the similarity of the individuals in the semantic space. In addition, the analysis includes random selection and traditional tournament selection. These selection functions were implemented on our Semantic Genetic Programming system, namely EvoDAG, which is inspired by the geometric genetic operators and tested on 30 classification problems with a variable number of samples, variables, and classes. The result indicated that the combination of accuracy and the random selection, in the negative tournament, produces the best combination, and the difference in performances between this combination and the tournament selection is statistically significant. Furthermore, we compare EvoDAG's performance using the selection heuristics against 18 classifiers that included traditional approaches as well as auto-machine-learning techniques. The results indicate that our proposal is competitive with state-of-art classifiers. Finally, it is worth to mention that EvoDAG is available as open source software.
Let us recall that Semantic GP uses the information in the target behavior, i.e., @math , to guide the search. Notably, Krawiec @cite_29 affirmed that aware semantic methods make search algorithms better informed. For example, Nguyen @cite_28 proposed Fitness Sharing, a technique that promotes dispersion and diversity of individuals. Their proposal consisted of calculating the individual fitness as @math , where @math is approximately equal to the number of individuals that behave similarly to individual @math .
@cite_28 @cite_29
2113192631 2043686990
This paper investigates the efficiency of using semantic and syntactic distance metrics in fitness sharing with Genetic Programming (GP). We modify the implementation of fitness sharing to speed up its execution, and used two distance metrics in calculating the distance between individuals in fitness sharing: semantic distance and syntactic distance. We applied fitness sharing with these two distance metrics to a class of real-valued symbolic regression. Experimental results show that using semantic distance in fitness sharing helps to significantly improve the performance of GP more frequently, and results in faster execution times than with the syntactic distance. Moreover, we also analyse the impact of the fitness sharing parameters on GP performance helping to indicate appropriate values for fitness sharing using a semantic distance metric. Geometric semantic genetic programming (GP) has recently attracted much attention. The key innovations are inducing a unimodal fitness landscape in the semantic space and providing a theoretical framework for designing geometric semantic operators. The geometric semantic operators aim to manipulate the semantics of programs by making a bounded semantic impact and generating child programs with similar or better behavior than their parents. These properties are shown to be highly related to a notable generalization improvement in GP. However, the potential ineffectiveness and difficulties in bounding the variations in these geometric operators still limits their positive effect on generalization. This paper attempts to further explore the geometry and search space of geometric operators to gain a greater generalization improvement in GP for symbolic regression. To this end, a new angle-driven selection operator and two new angle-driven geometric search operators are proposed. The angle-awareness brings new geometric properties to these geometric operators, which are expected to provide a greater leverage for approximating the target semantics in each operation, and more importantly, be resistant to overfitting. The experiments show that compared with two state-of-the-art geometric semantic operators, our angle-driven geometric operators not only drive the evolutionary process to fit the target semantics more efficiently but also improve the generalization performance. A further comparison between the evolved models shows that the new method generally produces simpler models with a much smaller size and is more likely to evolve toward the correct structure of the target models. In this paper, we propose a framework to study a general class of strategic behavior in voting, which we call vote operations. We prove the following theorem: if we fix the number of alternatives, generate @math votes i.i.d. according to a distribution @math , and let @math go to infinity, then for any @math , with probability at least @math , the minimum number of operations that are needed for the strategic individual to achieve her goal falls into one of the following four categories: (1) 0, (2) @math , (3) @math , and (4) @math . This theorem holds for any set of vote operations, any individual vote distribution @math , and any integer generalized scoring rule, which includes (but is not limited to) almost all commonly studied voting rules, e.g., approval voting, all positional scoring rules (including Borda, plurality, and veto), plurality with runoff, Bucklin, Copeland, maximin, STV, and ranked pairs. We also show that many well-studied types of strategic behavior fall under our framework, including (but not limited to) constructive destructive manipulation, bribery, and control by adding deleting votes, margin of victory, and minimum manipulation coalition size. Therefore, our main theorem naturally applies to these problems. Optimal search for targets with conditionally deterministic motion is investigated. The target motion takes place in @math , a copy of Euclidean n-space, and depends on a stochastic parameter @math which takes values in @math , another copy of Euclidean n-space. The target motion is deterministic given knowledge of @math . That is, there is a function @math , where T is a time interval, such that @math gives the target motion conditioned on @math . Search plans are specified by functions @math . A functional P is defined so that @math gives the probability of detecting the target by time t using plan @math .Let @math be the absolute value of the Jacobian of @math evaluated at x. If there exist functions @math and @math such that @math for @math , the target motion is called factorable Let $ _2 :T ... In recent years scholars have built maps of science by connecting the academic fields that cite each other, are cited together, or that cite a similar literature. But since scholars cannot always publish in the fields they cite, or that cite them, these science maps are only rough proxies for the potential of a scholar, organization, or country, to enter a new academic field. Here we use a large dataset of scholarly publications disambiguated at the individual level to create a map of science-or research space-where links connect pairs of fields based on the probability that an individual has published in both of them. We find that the research space is a significantly more accurate predictor of the fields that individuals and organizations will enter in the future than citation based science maps. At the country level, however, the research space and citations based science maps are equally accurate. These findings show that data on career trajectories-the set of fields that individuals have previously published in-provide more accurate predictors of future research output for more focalized units-such as individuals or organizations-than citation based science maps. We develop a Bayesian nonparametric approach to a general family of latent class problems in which individuals can belong simultaneously to multiple classes and where each class can be exhibited multiple times by an individual. We introduce a combinatorial stochastic process known as the negative binomial process ( @math ) as an infinite-dimensional prior appropriate for such problems. We show that the @math is conjugate to the beta process, and we characterize the posterior distribution under the beta-negative binomial process ( @math ) and hierarchical models based on the @math (the @math ). We study the asymptotic properties of the @math and develop a three-parameter extension of the @math that exhibits power-law behavior. We derive MCMC algorithms for posterior inference under the @math , and we present experiments using these algorithms in the domains of image segmentation, object recognition, and document analysis. We consider the problem of online learning in the linear contextual bandits setting, but in which there are also strong individual fairness constraints governed by an unknown similarity metric. These constraints demand that we select similar actions or individuals with approximately equal probability (arXiv:1104.3913), which may be at odds with optimizing reward, thus modeling settings where profit and social policy are in tension. We assume we learn about an unknown Mahalanobis similarity metric from only weak feedback that identifies fairness violations, but does not quantify their extent. This is intended to represent the interventions of a regulator who "knows unfairness when he sees it" but nevertheless cannot enunciate a quantitative fairness metric over individuals. Our main result is an algorithm in the adversarial context setting that has a number of fairness violations that depends only logarithmically on @math , while obtaining an optimal @math regret bound to the best fair policy. We investigate the minimization of Newton's functional for the problem of the body of minimal resistance with maximal height @math cite butt in the class of convex developable functions defined in a disc. This class is a natural candidate to find a (non-radial) minimizer in accordance with the results of cite lrp2. We prove that the minimizer in this class has a minimal set in the form of a regular polygon with @math sides centered in the disc, and numerical experiments indicate that the natural number @math is a non-decreasing function of @math . The corresponding functions all achieve a lower value of the functional than the optimal radially symmetric function with the same height @math . The subspace approximation problem Subspace( @math , @math ) asks for a @math -dimensional linear subspace that fits a given set of points optimally, where the error for fitting is a generalization of the least squares fit and uses the @math norm instead. Most of the previous work on subspace approximation has focused on small or constant @math and @math , using coresets and sampling techniques from computational geometry. In this paper, extending another line of work based on convex relaxation and rounding, we give a polynomial time algorithm, , with the approximation guarantee roughly @math , where @math is the @math -th moment of a standard normal random variable N(0,1). We show that the convex relaxation we use has an integrality gap (or "rank gap") of @math , for any constant @math . Finally, we show that assuming the Unique Games Conjecture, the subspace approximation problem is hard to approximate within a factor better than @math , for any constant @math . This work provides tight upper- and lower-bounds for the problem of mean estimation under @math -differential privacy in the local model, when the input is composed of @math i.i.d. drawn samples from a normal distribution with variance @math . Our algorithms result in a @math -confidence interval for the underlying distribution's mean @math of length @math . In addition, our algorithms leverage binary search using local differential privacy for quantile estimation, a result which may be of separate interest. Moreover, we prove a matching lower-bound (up to poly-log factors), showing that any one-shot (each individual is presented with a single query) local differentially private algorithm must return an interval of length @math . Given samples from an unknown multivariate distribution @math , is it possible to distinguish whether @math is the product of its marginals versus @math being far from every product distribution? Similarly, is it possible to distinguish whether @math equals a given distribution @math versus @math and @math being far from each other? These problems of testing independence and goodness-of-fit have received enormous attention in statistics, information theory, and theoretical computer science, with sample-optimal algorithms known in several interesting regimes of parameters. Unfortunately, it has also been understood that these problems become intractable in large dimensions, necessitating exponential sample complexity. Motivated by the exponential lower bounds for general distributions as well as the ubiquity of Markov Random Fields (MRFs) in the modeling of high-dimensional distributions, we initiate the study of distribution testing on structured multivariate distributions, and in particular the prototypical example of MRFs: the Ising Model. We demonstrate that, in this structured setting, we can avoid the curse of dimensionality, obtaining sample and time efficient testers for independence and goodness-of-fit. One of the key technical challenges we face along the way is bounding the variance of functions of the Ising model. We propose a new randomized coordinate descent method for minimizing the sum of convex functions each of which depends on a small number of coordinates only. Our method (APPROX) is simultaneously Accelerated, Parallel, and PROXimal; this is the first time such a method is proposed. In the special case when the number of processors is equal to the number of coordinates, the method converges at the rate @math , where @math is the iteration counter, @math is a data-weighted average degree of separability of the loss function, @math is the average of Lipschitz constants associated with the coordinates and individual functions in the sum, and @math is the distance of the initial point from the minimizer. We show that the method can be implemented without the need to perform full-dimensional vector operations, which is the major bottleneck of accelerated coordinate descent. The fact that the method depends on the average degree of separability, and not on the maximum degree... We study supercritical branching Brownian motion on the real line starting at the origin and with constant drift @math . At the point @math , we add an absorbing barrier, i.e. individuals touching the barrier are instantly killed without producing offspring. It is known that there is a critical drift @math , such that this process becomes extinct almost surely if and only if @math . In this case, if @math denotes the number of individuals absorbed at the barrier, we give an asymptotic for @math as @math goes to infinity. If @math and the reproduction is deterministic, this improves upon results of [L. Addario-Berry and N. Broutin (2009), http: arxiv.org abs 0908.1083v1 ] and [E. A " dekon (2009), http: arxiv.org abs 0911.0877v1 ] on a conjecture by David Aldous about the total progeny of a branching random walk. The main technique used in the proofs is analysis of the generating function of @math near its singular point @math , based on classical results on some complex differential equations. Abstract Delay discounting (DD) is a measure of the degree to which an individual is driven by immediate gratification vs. the prospect of larger, but delayed, rewards. Because of hypothesized parallels between drug addiction and obesity, and reports of increased delay discounting in drug-dependent individuals, we hypothesized that obese individuals would show higher rates of discounting than controls. Obese and healthy-weight age-matched participants of both sexes completed two versions of a DD of money task, allowing us to calculate how subjective value of @math 50,000 declined as delay until hypothetical delivery increased from 2 weeks to 10 years. On both tasks, obese women (N = 29) showed greater delay discounting than control women did (N = 26; P values In this work we analyze the sample complexity of classification by differentially private algorithms. Differential privacy is a strong and well-studied notion of privacy introduced by (2006) that ensures that the output of an algorithm leaks little information about the data point provided by any of the participating individuals. Sample complexity of private PAC and agnostic learning was studied in a number of prior works starting with (, 2008) but a number of basic questions still remain open, most notably whether learning with privacy requires more samples than learning without privacy. We show that the sample complexity of learning with (pure) differential privacy can be arbitrarily higher than the sample complexity of learning without the privacy constraint or the sample complexity of learning with approximate differential privacy. Our second contribution and the main tool is an equivalence between the sample complexity of (pure) differentially private learning of a concept class @math (or @math ) and the randomized one-way communication complexity of the evaluation problem for concepts from @math . Using this equivalence we prove the following bounds: 1. @math , where @math is the Littlestone's (1987) dimension characterizing the number of mistakes in the online-mistake-bound learning model. Known bounds on @math then imply that @math can be much higher than the VC-dimension of @math . 2. For any @math , there exists a class @math such that @math but @math . 3. For any @math , there exists a class @math such that the sample complexity of (pure) @math -differentially private PAC learning is @math but the sample complexity of the relaxed @math -differentially private PAC learning is @math . This resolves an open problem of (2013b). We propose a simple solution to use a single Neural Machine Translation (NMT) model to translate between multiple languages. Our solution requires no change in the model architecture from our base system but instead introduces an artificial token at the beginning of the input sentence to specify the required target language. The rest of the model, which includes encoder, decoder and attention, remains unchanged and is shared across all languages. Using a shared wordpiece vocabulary, our approach enables Multilingual NMT using a single model without any increase in parameters, which is significantly simpler than previous proposals for Multilingual NMT. Our method often improves the translation quality of all involved language pairs, even while keeping the total number of model parameters constant. On the WMT'14 benchmarks, a single multilingual model achieves comparable performance for English @math French and surpasses state-of-the-art results for English @math German. Similarly, a single multilingual model surpasses state-of-the-art results for French @math English and German @math English on WMT'14 and WMT'15 benchmarks respectively. On production corpora, multilingual models of up to twelve language pairs allow for better translation of many individual pairs. In addition to improving the translation quality of language pairs that the model was trained with, our models can also learn to perform implicit bridging between language pairs never seen explicitly during training, showing that transfer learning and zero-shot translation is possible for neural translation. Finally, we show analyses that hints at a universal interlingua representation in our models and show some interesting examples when mixing languages. We consider the problem of learning an unknown function @math on the @math -dimensional sphere with respect to the square loss, given i.i.d. samples @math where @math is a feature vector uniformly distributed on the sphere and @math . We study two popular classes of models that can be regarded as linearizations of two-layers neural networks around a random initialization: (RF) The random feature model of Rahimi-Recht; (NT) The neural tangent kernel model of Jacot-Gabriel-Hongler. Both these approaches can also be regarded as randomized approximations of kernel ridge regression (with respect to different kernels), and hence enjoy universal approximation properties when the number of neurons @math diverges, for a fixed dimension @math . We prove that, if both @math and @math are large, the behavior of these models is instead remarkably simpler. If @math , then RF performs no better than linear regression with respect to the raw features @math , and NT performs no better than linear regression with respect to degree-one and two monomials in the @math . More generally, if @math then RF fits at most a degree- @math polynomial in the raw features, and NT fits at most a degree- @math polynomial. Let @math be @math -variate polynomials with rational coefficients of maximum degree @math and let @math be the set of common complex solutions of @math . We give an algorithm which, up to some regularity assumptions on @math , computes an exact representation of the global infimum @math of the restriction of the map @math to @math , i.e., a univariate polynomial vanishing at @math and an isolating interval for @math . Furthermore, it decides whether @math is reached, and if so, it returns @math such that @math . This algorithm is probabilistic. It makes use of the notion of polar varieties. Its complexity is essentially cubic in @math and linear in the complexity of evaluating the input. This fits within the best known deterministic complexity class @math . We report on some practical experiments of a first implementation that is available as a Maple package. It appears that it ca... Semantic genetic programming is a recent, rapidly growing trend in Genetic Programming (GP) that aims at opening the 'black box' of the evaluation function and make explicit use of more information on program behavior in the search. In the most common scenario of evaluating a GP program on a set of input-output examples (fitness cases), the semantic approach characterizes program with a vector of outputs rather than a single scalar value (fitness). The past research on semantic GP has demonstrated that the additional information obtained in this way facilitates designing more effective search operators. In particular, exploiting the geometric properties of the resulting semantic space leads to search operators with attractive properties, which have provably better theoretical characteristics than conventional GP operators. This in turn leads to dramatic improvements in experimental comparisons. The aim of the tutorial is to give a comprehensive overview of semantic methods in genetic programming, illustrate in an accessible way a formal geometric framework for program semantics to design provably good mutation and crossover operators for traditional GP problem domains, and to analyze rigorously their performance (runtime analysis). A number of real-world applications of this framework will be also presented. Other promising emerging approaches to semantics in GP will be reviewed. In particular, the recent developments in the behavioral programming, which aims at characterizing the entire program behavior (and not only program outputs) will be covered as well. Current challenges and future trends in semantic GP will be identified and discussed. Selected methods and concepts will be accompanied with live software demonstrations. Also, efficient implementation of semantic search operators may be challenging. We will illustrate very efficient, concise and elegant implementations of these operators, which are available for download from the web. We analyze and solve a game in which a player chooses which of several Markov chains to advance, with the object of minimizing the expected time (or cost) for one of the chains to reach a target state. The solution entails computing (in polynomial time) a function @math ---a variety of "Gittins index"---on the states of the individual chains, the minimization of which produces an optimal strategy. It turns out that @math is a useful cousin of the expected hitting time of a Markov chain but is defined, for example, even for random walks on infinite graphs. We derive the basic properties of @math and consider its values in some natural situations.