10859249
Making use of population information in evolutionary artificial neural networks
This paper is concerned with the simultaneous evolution of artificial neural network (ANN) architectures and weights. The current practice in evolving ANN's is to choose the best ANN in the last generation as the final result. This paper proposes a different approach to form the final result by combining all the individuals in the last generation in order to make best use of all the information contained in the whole population. This approach regards a population of ANN's as an ensemble and uses a combination method to integrate them. Although there has been some work on integrating ANN modules, little has been done in evolutionary learning to make best use of its population information. Four linear combination methods have been investigated in this paper to illustrate our ideas. Three real-world data sets have been used in our experimental studies, which show that the recursive least-square (RLS) algorithm always produces an integrated system that outperforms the best individual. The results confirm that a population contains more information than a single individual. Evolutionary learning should exploit such information to improve generalization of learned systems.
18179447
Towards Designing Neural Network Ensembles by Evolution
This paper proposes a co-evolutionary learning system, i.e., CELS, to design neural network (NN) ensembles. CELS addresses the issue of automatic determination of the number of individual NNs in an ensemble and the exploitation of the interaction between individual NN design and combination. The idea of CELS is to encourage different individual NNs in the ensemble to learn different parts or aspects of the training data so that the ensemble can learn the whole training data better. The cooperation and specialisation among different individual NNs are considered during the individual NN design. This provides an opportunity for different NNs to interact with each other and to specialise. Experiments on two real-world problems demonstrate that CELS can produce NN ensembles with good generalisation ability.
15370058
Evolutionary programming made faster
Evolutionary programming (EP) has been applied with success to many numerical and combinatorial optimization problems in recent years. EP has rather slow convergence rates, however, on some function optimization problems. In the paper, a "fast EP" (FEP) is proposed which uses a Cauchy instead of Gaussian mutation as the primary search operator. The relationship between FEP and classical EP (CEP) is similar to that between fast simulated annealing and the classical version. Both analytical and empirical studies have been carried out to evaluate the performance of FEP and CEP for different function optimization problems. The paper shows that FEP is very good at search in a large neighborhood while CEP is better at search in a small local neighborhood. For a suite of 23 benchmark problems, FEP performs much better than CEP for multimodal functions with many local minima while being comparable to CEP in performance for unimodal and multimodal functions with only a few local minima. The paper also shows the relationship between the search step size and the probability of finding a global optimum and thus explains why FEP performs better than CEP on some functions but not on others. In addition, the importance of the neighborhood size and its relationship to the probability of finding a near-optimum is investigated. Based on these analyses, an improved FEP (IFEP) is proposed and tested empirically. This technique mixes different search operators (mutations). The experimental results show that IFEP performs better than or as well as the better of FEP and CEP for most benchmark problems tested.
45330274
Time Series Prediction by Using Negatively Correlated Neural Networks
Negatively correlated neural networks (NCNNs) have been proposed to design neural network (NN) ensembles [1]. The idea of NCNNs is to encourage different individual NNs in the ensemble to learn different parts or aspects of a training data so that the ensemble can learn the whole training data better. The cooperation and specialisation among different individual NNs are considered during the individual NN design. This provides an opportunity for different NNs to interact with each other and to specialise. In this paper, NCNNs are applied to two time series prediction problems (i.e., the Mackey-Glass differential equation and the chlorophyll-a prediction in Lake Kasumigaura). The experimental results show that NCNNs can produce NN ensembles with good generalisation ability.
16112032
Fast Evolution Strategies
Evolution strategies are a class of general optimisation algorithms which are applicable to functions that are multimodal, non-differentiable, or even discontinuous. Although recombination operators have been introduced into evolution strategies, their primary search operator is still mutation. Classical evolution strategies rely on Gaussian mutations. A new mutation operator based on the Cauchy distribution is proposed in this paper. It is shown empirically that the new evolution strategy based on Cauchy mutation outperforms the classical evolution strategy on most of the 23 benchmark problems tested in this paper. These results, along with those obtained by fast evolutionary programming
9804135
EPNet for Chaotic Time-Series Prediction
EPNet is an evolutionary system for automatic design of artificial neural networks (ANNs) [1, 2, 3]. Unlike most previous methods on evolving ANNs, EPNet puts its emphasis on evolving ANN'S behaviours rather than circuitry. The parsimony of evolved ANNs is encouraged by the sequential application of architectural mutations. In this paper, EP Net is applied to a couple of chaotic time-series prediction problems (i.e., the Mackey-Glass differential equation and the logistic map). The experimental results show that EPNet can produce very compact ANNs with good prediction ability in comparison with other algorithms.
5651676
Segmentation Conditional Random Fields (SCRFs): A New Approach for Protein Fold Recognition
Protein fold recognition is an important step towards understanding protein three-dimensional structures and their functions. A conditional graphical model, i.e. segmentation conditional random fields (SCRFs), is proposed to solve the problem. In contrast to traditional graphical models such as hidden markov model (HMM), SCRFs follow a discriminative approach. It has the flexibility to include overlapping or long-range interaction features over the whole sequence, as well as global optimally solutions for the parameters. On the other hand, the segmentation setting in SCRFs makes its graphical structures intuitively similar to the protein 3-D structures and more importantly, provides a framework to model the long-range interactions directly. 
 
Our model is applied to predict the parallel β-helix fold, an important fold in bacterial infection of plants and binding of antigens. The cross-family validation shows that SCRFs not only can score all known β-helices higher than non β-helices in Protein Data Bank, but also demonstrate more success in locating each rung in the known β-helix proteins than BetaWrap, a state-of-the-art algorithm for predicting β-helix fold, and HMMER, a general motif detection algorithm based on HMM. Applying our prediction model to Uniprot database, we hypothesize previously unknown β-helices.
10720584
Graph-Based Semi-Supervised Learning as a Generative Model
This paper proposes and develops a new graph-based semi-supervised learning method. Different from previous graph-based methods that are based on discriminative models, our method is essentially a generative model in that the class conditional probabilities are estimated by graph propagation and the class priors are estimated by linear regression. Experimental results on various datasets show that the proposed method is superior to existing graph-based semi-supervised learning methods, especially when the labeled subset alone proves insufficient to estimate meaningful class priors.
30635259
On global exponential stability of generalized stochastic neural networks with mixed time-delays
Abstract This paper is concerned with the global exponential stability analysis problem for a general class of stochastic neural networks with mixed time-delays. The mixed time-delays under consideration comprise both the discrete time-varying delays and the distributed time-delays. The main purpose of this paper is to establish easily verifiable conditions under which the delayed stochastic neural network is exponentially stable in the mean square in the presence of both the discrete and distributed delays. By employing a new Lyapunov–Krasovskii functional and conducting stochastic analysis, a linear matrix inequality (LMI) approach is developed to derive the criteria of the exponential stability. Furthermore, the main results are specialized to deal with the analysis problem for the global asymptotic stability within the same LMI framework. The proposed criteria can be readily checked by using some standard numerical packages such as the Matlab LMI toolbox. A simple example is provided to demonstrate the effectiveness and applicability of the proposed testing criteria.
6222461
Approximate VCCs: a new characterization of multimedia workloads for system-level MpSoC design
System-level design methods specifically targeted towards multimedia applications have recently received a lot of attention. Multimedia workloads are known to have a high degree of variability. Therefore, designs based on a worst-case analysis of such workloads tend of be overly pessimistic. We address this issue by introducing a new concept called approximate variability characterization curves (or approximate VCCs), to characterize the "average-case" behavior of multimedia workloads in a parameterized fashion. Since most multimedia applications only have soft real-time constraints, it is often possible to tolerate a small amount of performance degradation. By allowing such small degradations in the performance, large amounts of resource savings are possible. The concept of approximate VCCs that we present in this paper allows a designer to quantitatively account for the performance degradation and the associated resource savings. We illustrate this using two typical system design cases.
11751960
Discovering auxiliary information for incremental computation
This paper presents program analyses and transformations that discover a general class of auxiliary information for any incremental computation problem. Combining these techniques with previous techniques for caching intermediate results, we obtain a systematic approach that transforms nonincremental programs into efficient incremental programs that use and maintain useful auxiliary information as well as useful intermediate results. The use of auxiliary information allows us to achieve a greater degree of incrementality than otherwise possible. Applications of the approach include strength reduction in optimizing compilers and finite differencing in transformational programming.