28337697
Trends in Neural Computation
Hyperbolic Function Networks for Pattern Classification.- Variable Selection for the Linear Support Vector Machine.- Selecting Data for Fast Support Vector Machines Training.- Universal Approach to Study Delayed Dynamical Systems.- A Hippocampus-Neocortex Model for Chaotic Association.- Latent Attractors: A General Paradigm for Context-Dependent Neural Computation.- Learning Mechanisms in Networks of Spiking Neurons.- GTSOM: Game Theoretic Self-organizing Maps.- How to Generate Different Neural Networks.- A Gradient-Based Forward Greedy Algorithm for Space Gaussian Process Regression.- An Evolved Recurrent Neural Network and Its Application.- A Min-Max Modular Network with Gaussian-Zero-Crossing Function.- Combining Competitive Learning Networks of Various Representations for Sequential Data Clustering.- Modular Neural Networks and Their Applications in Biometrics.- Performance Analysis of Dynamic Cell Structures.- Short Term Electric Load Forecasting: A Tutorial.- Performance Improvement for Formation-Keeping Control Using a Neural Network HJI Approach.- A Robust Blind Neural Equalizer Based on Higher-Order Cumulants.- The Artificial Neural Network Applied to Servo Control System.- Robot Localization Using Vision.
15908834
Ensemble Learning with Active Data Selection for Semi-Supervised Pattern Classification
Unlike traditional pattern classification, semi-supervised learning provides a novel technique to make use of both labeled and unlabeled data for improving the performance of classification. In general, there are two critical issues for semi-supervised learning of discriminative classifiers; i.e., how to create an initial classifier of a good generalization capability with the limited labeled data and the how to make an effective use of unlabeled data without degradation of the established classifier. To tackle two aforementioned problems, we propose an ensemble learning approach based on a recent active data selection strategy, where ensemble learning would yield good generalization and active data selection tends to choose the unlabeled data more likely resulting in an improvement during semi-supervised learning. By using an ensemble of K-NN classifiers, we demonstrate the effectiveness of our approach on a synthetic data classification and a facial expression recognition tasks.
5764451
Regularized Boost for Semi-Supervised Learning
Semi-supervised inductive learning concerns how to learn a decision rule from a data set containing both labeled and unlabeled data. Several boosting algorithms have been extended to semi-supervised learning with various strategies. To our knowledge, however, none of them takes local smoothness constraints among data into account during ensemble learning. In this paper, we introduce a local smoothness regularizer to semi-supervised boosting algorithms based on the universal optimization framework of margin cost functionals. Our regularizer is applicable to existing semi-supervised boosting algorithms to improve their generalization and speed up their training. Comparative results on synthetic, benchmark and real world tasks demonstrate the effectiveness of our local smoothness regularizer. We discuss relevant issues and relate our regularizer to previous work.
6083596
Improved learning algorithms for mixture of experts in multiclass classification
Mixture of experts (ME) is a modular neural network architecture for supervised learning. A double-loop Expectation-Maximization (EM) algorithm has been introduced to the ME architecture for adjusting the parameters and the iteratively reweighted least squares (IRLS) algorithm is used to perform maximization in the inner loop [Jordan, M.I., Jacobs, R.A. (1994). Hierarchical mixture of experts and the EM algorithm, Neural Computation, 6(2), 181-214]. However, it is reported in literature that the IRLS algorithm is of instability and the ME architecture trained by the EM algorithm, where IRLS algorithm is used in the inner loop, often produces the poor performance in multiclass classification. In this paper, the reason of this instability is explored. We find out that due to an implicitly imposed incorrect assumption on parameter independence in multiclass classification, an incomplete Hessian matrix is used in that IRLS algorithm. Based on this finding, we apply the Newton-Raphson method to the inner loop of the EM algorithm in the case of multiclass classification, where the exact Hessian matrix is adopted. To tackle the expensive computation of the Hessian matrix and its inverse, we propose an approximation to the Newton-Raphson algorithm based on a so-called generalized Bernoulli density. The Newton-Raphson algorithm and its approximation have been applied to synthetic data, benchmark, and real-world multiclass classification tasks. For comparison, the IRLS algorithm and a quasi-Newton algorithm called BFGS have also been applied to the same tasks. Simulation results have shown that the use of the proposed learning algorithms avoids the instability problem and makes the ME architecture produce good performance in multiclass classification. In particular, our approximation algorithm leads to fast learning. In addition, the limitation of our approximation algorithm is also empirically investigated in this paper.
15381679
A Modular Neural Network Architecture for Pattern Classification Based on Different Feature Sets
We propose a novel connectionist method for the use of different feature sets in pattern classification. Unlike traditional methods, e.g., combination of multiple classifiers and use of a composite feature set, our method copes with the problem based on an idea of soft competition on different feature sets developed in our earlier work. An alternative modular neural network architecture is proposed to provide a more effective implementation of soft competition on different feature sets. The proposed architecture is interpreted as a generalized finite mixture model and, therefore, parameter estimation is treated as a maximum likelihood problem. An EM algorithm is derived for parameter estimation and, moreover, a model selection method is proposed to fit the proposed architecture to a specific problem. Comparative results are presented for the real world problem of speaker identification.
64402095
Trends in Neural Computation (Studies in Computational Intelligence)
3106142
MATLAB Simulation and Comparison of Zhang Neural Network and Gradient Neural Network for Online Solution of Linear Time-Varying Matrix Equation AXB-C=0
Different from gradient neural networks (GNN), a special kind ofrecurrent neural networks has been proposed recently by Zhanget alfor solving online linear matrix equations withtime-varying coefficients. Such recurrent neural networks, designedbased on a matrix-valued error-function, could achieve globalexponential convergence when solving online time-varying problemsin comparison with gradient neural networks. This paperinvestigates the MATLAB simulation of Zhang neural networks (ZNN)for real-time solution of linear time-varying matrix equationAXB- C= 0. Gradient neural networks aresimulated and compared as well. Simulation results substantiate thetheoretical analysis and efficacy of ZNN on linear time-varyingmatrix equation solving.
31501013
Preconditioning and convergence in the right norm
16063876
How to play a coloring game against a color-blind adversary
We study the problem of conflict-free (CF) coloring of a set of points in the plane, in an online fashion, with respect to halfplanes, nearly-equal axis-parallel rectangles, and congruent disks. As a warm-up exercise, the online CF coloring of points on the line with respect to intervals is also considered. We present randomized algorithms in the oblivious adversary model, where the adversary does not see the colors used. For the problems considered, the algorithms always produce valid CF colorings, and use O(logn) colors with high probability (these bounds are optimal in the worst case). Our randomized online algorithms are considerably simpler than previous algorithms for this problem and use fewer colors.We also present a deterministic algorithm for the CF coloring of points in the plane with respect to nearly-equal axis-parallel rectangles, using O(polylog(n)) colors. This is the first efficient deterministic online CF coloring algorithm for this problem.
14977777
Zhang neural network without using time-derivative information for constant and time-varying matrix inversion
To obtain the inverses of time-varying matrices in real time, a special kind of recurrent neural networks has recently been proposed by Zhang et al. It is proved that such a Zhang neural network (ZNN) could globally exponentially converge to the exact inverse of a given time-varying matrix. To find out the effect of time-derivative term on global convergence as well as for easier hardware-implementation purposes, the ZNN model without exploiting time-derivative information is investigated in this paper for inverting online matrices. Theoretical results of both constant matrix inversion case and time-varying matrix inversion case are presented for comparative and illustrative purposes. In order to substantiate the presented theoretical results, computer-simulation results are shown, which demonstrate the importance of time derivative term of given matrices on the exact convergence of ZNN model to time-varying matrix inverses.
13249966
Prediction of integral membrane protein type by collocated hydrophobic amino acid pairs
A computational model, IMP-TYPE, is proposed for the classification of five types of integral membrane proteins from protein sequence. The proposed model aims not only at providing accurate predictions but most importantly it incorporates interesting and transparent biological patterns. When contrasted with the best-performing existing models, IMP-TYPE reduces the error rates of these methods by 19 and 34% for two out-of-sample tests performed on benchmark datasets. Our empirical evaluations also show that the proposed method provides even bigger improvements, i.e., 29 and 45% error rate reductions, when predictions are performed for sequences that share low (40%) identity with sequences from the training dataset. We also show that IMP-TYPE can be used in a standalone mode, i.e., it duplicates significant majority of correct predictions provided by other leading methods, while providing additional correct predictions which are incorrectly classified by the other methods. Our method computes predictions using a Support Vector Machine classifier that takes feature-based encoded sequence as its input. The input feature set includes hydrophobic AA pairs, which were selected by utilizing a consensus of three feature selection algorithms. The hydrophobic residues that build up the AA pairs used by our method are shown to be associated with the formation of transmembrane helices in a few recent studies concerning integral membrane proteins. Our study also indicates that Met and Phe display a certain degree of hydrophobicity, which may be more crucial than their polarity or aromaticity when they occur in the transmembrane segments. This conclusion is supported by a recent study on potential of mean force for membrane protein folding and a study of scales for membrane propensity of amino acids.