18179447
Towards Designing Neural Network Ensembles by Evolution
This paper proposes a co-evolutionary learning system, i.e., CELS, to design neural network (NN) ensembles. CELS addresses the issue of automatic determination of the number of individual NNs in an ensemble and the exploitation of the interaction between individual NN design and combination. The idea of CELS is to encourage different individual NNs in the ensemble to learn different parts or aspects of the training data so that the ensemble can learn the whole training data better. The cooperation and specialisation among different individual NNs are considered during the individual NN design. This provides an opportunity for different NNs to interact with each other and to specialise. Experiments on two real-world problems demonstrate that CELS can produce NN ensembles with good generalisation ability.
37420469
Getting most out of evolutionary approaches
Evolutionary algorithms (EAs) have been widely used in evolvable hardware. The very term, evolvable hardware, reflects the importance and omnitude of EAs in this field. However, EAs have primarily been used as an optimisation or search tool, which can explore a large and complex space. While success has been demonstrated by EAs in exploring unconventional designs that are hard to reach by human experts, it is interesting to ask the question whether we have fully used all the potentialities of EAs. We argue in this paper that there is rich information in a population which can and should be exploited. The classical approach of evolving the best individual in a population may not be the best one. A truly population-based approach that emphasizes population rather than the best individual can often bring in several important benefits to evolvable hardware, including efficiency, accuracy, adaptiveness, and fault-tolerance.
16112032
Fast Evolution Strategies
Evolution strategies are a class of general optimisation algorithms which are applicable to functions that are multimodal, non-differentiable, or even discontinuous. Although recombination operators have been introduced into evolution strategies, their primary search operator is still mutation. Classical evolution strategies rely on Gaussian mutations. A new mutation operator based on the Cauchy distribution is proposed in this paper. It is shown empirically that the new evolution strategy based on Cauchy mutation outperforms the classical evolution strategy on most of the 23 benchmark problems tested in this paper. These results, along with those obtained by fast evolutionary programming
15370058
Evolutionary programming made faster
Evolutionary programming (EP) has been applied with success to many numerical and combinatorial optimization problems in recent years. EP has rather slow convergence rates, however, on some function optimization problems. In the paper, a "fast EP" (FEP) is proposed which uses a Cauchy instead of Gaussian mutation as the primary search operator. The relationship between FEP and classical EP (CEP) is similar to that between fast simulated annealing and the classical version. Both analytical and empirical studies have been carried out to evaluate the performance of FEP and CEP for different function optimization problems. The paper shows that FEP is very good at search in a large neighborhood while CEP is better at search in a small local neighborhood. For a suite of 23 benchmark problems, FEP performs much better than CEP for multimodal functions with many local minima while being comparable to CEP in performance for unimodal and multimodal functions with only a few local minima. The paper also shows the relationship between the search step size and the probability of finding a global optimum and thus explains why FEP performs better than CEP on some functions but not on others. In addition, the importance of the neighborhood size and its relationship to the probability of finding a near-optimum is investigated. Based on these analyses, an improved FEP (IFEP) is proposed and tested empirically. This technique mixes different search operators (mutations). The experimental results show that IFEP performs better than or as well as the better of FEP and CEP for most benchmark problems tested.
2748390
Ensemble structure of evolutionary artificial neural networks
Evolutionary artificial neural networks (EANNs) refer to a special class of artificial neural networks (ANNs) in which evolution is another fundamental form of adaptation in addition to learning. Evolution can be introduced at various levels of ANNs. It can be used to evolve weights, architectures, and learning parameters and rules. The paper is concerned with the evolution of ANN architectures, where an evolutionary algorithm is used to evolve a population of ANNs. The current practice in evolving ANNs is to choose the best ANN in the last population as the final result. The paper proposes a novel approach to form the final result by combining all the individuals in the last generation in order to make best use of all the information contained in the whole population. This approach regards a population of ANNs as an ensemble of ANNs and use a method to combine them. We have used four simple methods in our computational studies. The first is the majority voting method. The second and third are linear combination methods over the whole population. The fourth is a linear combination method over a subset of the whole population. The near optimal subset is obtained by a genetic algorithm search. Our experiments have shown that all four methods have produced better results than those produced by the single best individual.
9804135
EPNet for Chaotic Time-Series Prediction
EPNet is an evolutionary system for automatic design of artificial neural networks (ANNs) [1, 2, 3]. Unlike most previous methods on evolving ANNs, EPNet puts its emphasis on evolving ANN'S behaviours rather than circuitry. The parsimony of evolved ANNs is encouraged by the sequential application of architectural mutations. In this paper, EP Net is applied to a couple of chaotic time-series prediction problems (i.e., the Mackey-Glass differential equation and the logistic map). The experimental results show that EPNet can produce very compact ANNs with good prediction ability in comparison with other algorithms.
14275207
The Architectural Design of FRUIT: A Family of Retargetable User Interface Tools
Forms-based user interfaces are widely used means for human-computer interaction. However, it is difficult to exploit software reuse in forms-based systems because the systems tend to be dependent upon the particular display technologies used. This paper introduces the Family of Retargetable User Interface Tools (FRUIT) that aims to solve this problem. FRUIT adapts the device-independent XForms technology to deliver a family of products capable of supporting a variety of display technologies. This paper presents the architecture for the FRUIT product line and illustrates the design methodology used—a methodology based on stepwise refinement and commonality and variability analysis; it also describes two members of the FRUIT family and shows how their designs and implementations are refinements of the family architecture.
14501514
Semi-supervised Multi-label Learning by Constrained Non-negative Matrix Factorization
We present a novel framework for multi-label learning that explicitly addresses the challenge arising from the large number of classes and a small size of training data. The key assumption behind this work is that two examples tend to have large overlap in their assigned class memberships if they share high similarity in their input patterns. We capitalize this assumption by first computing two sets of similarities, one based on the input patterns of examples, and the other based on the class memberships of the examples. We then search for the optimal assignment of class memberships to the unlabeled data that minimizes the difference between these two sets of similarities. The optimization problem is formulated as a constrained Non-negative Matrix Factorization (NMF) problem, and an algorithm is presented to efficiently find the solution. Compared to the existing approaches for multi-label learning, the proposed approach is advantageous in that it is able to explore both the unlabeled data and the correlation among different classes simultaneously. Experiments with text categorization show that our approach performs significantly better than several state-of-the-art classification techniques when the number of classes is large and the size of training data is small.
1850975
Using classic problems to teach Java framework design
Abstract All programmers should understand the concept of software families and know the techniques for constructing them. This paper suggests that classic problems, such as well-known algorithms and data structures, are good sources for examples to use in a study of software family design. The paper describes two case studies that can be used to introduce students in a Java software design course to the construction of software families using software frameworks. The first is the family of programs that use the well-known divide and conquer algorithmic strategy. The second is the family of programs that carry out traversals of binary trees.
24959035
A Novel Constraint-Based Approach to Online Graphics Recognition
Online graphics recognition has become the key problem for pen-based user interface on small screen devices, such as PDA and Tablet PC. In this paper, a novel constraint-based approach to online graphics recognition is proposed. The key idea of our approach is that when the user is drawing a graphic object, the system can extract the constraints between primitives and basic shapes from the object and use these constraints to retrieve similar graphic objects from the database at run time. The user can then choose the standard object from the ranked list of results to replace his sketches before he finishes drawing all strokes of the object. For this purpose, we summarize three types of primitives and several types of basic shapes as the basic components of a graphic object. We also define a set of constraints between primitives and basic shapes to represent their structural relations. The algorithms for online constraint extraction and graphics recognition are also presented. Experimental results show that our approach is efficient for online graphics recognition and effective for improving the user’s productivity.
15563828
Self-similarity and long range dependence on the internet: a second look at the evidence, origins and implications
In this paper we critically reexamine some of the long standing beliefs regarding self-similarity and long range dependence (LRD) on the Internet. Power law tails have been conjectured to be a cause of LRD. In this paper, we reexamine the claims regarding heavy tails. We first examine the generative models for the heavy tail phenomena, both in terms of the fragility of some proposed mechanisms to modeling perturbations as well as the weak statistical evidence for the mechanisms. Next, we take a look at some of the implications of LRD in key performance aspects of Internet algorithms. Finally, we present an alternative model explaining the LRD phenomena of Internet traffic. We argue that the multiple time-scale nature of the generation of traffic and transport protocols make the observation of LRD inevitable.