3182057
A method of combining multiple probabilistic classifiers through soft competition on different feature sets
Abstract A novel method is proposed for combining multiple probabilistic classifiers on different feature sets. In order to achieve the improved classification performance, a generalized finite mixture model is proposed as a linear combination scheme and implemented based on radial basis function networks. In the linear combination scheme, soft competition on different feature sets is adopted as an automatic feature rank mechanism so that different feature sets can be always simultaneously used in an optimal way to determine linear combination weights. For training the linear combination scheme, a learning algorithm is developed based on Expectation–Maximization (EM) algorithm. The proposed method has been applied to a typical real-world problem, viz., speaker identification, in which different feature sets often need consideration simultaneously for robustness. Simulation results show that the proposed method yields good performance in speaker identification.
15908834
Ensemble Learning with Active Data Selection for Semi-Supervised Pattern Classification
Unlike traditional pattern classification, semi-supervised learning provides a novel technique to make use of both labeled and unlabeled data for improving the performance of classification. In general, there are two critical issues for semi-supervised learning of discriminative classifiers; i.e., how to create an initial classifier of a good generalization capability with the limited labeled data and the how to make an effective use of unlabeled data without degradation of the established classifier. To tackle two aforementioned problems, we propose an ensemble learning approach based on a recent active data selection strategy, where ensemble learning would yield good generalization and active data selection tends to choose the unlabeled data more likely resulting in an improvement during semi-supervised learning. By using an ensemble of K-NN classifiers, we demonstrate the effectiveness of our approach on a synthetic data classification and a facial expression recognition tasks.
34883067
Personalized News Reading via Hybrid Learning
In this paper, we present a personalized news reading prototype where latest news articles published by various on-line news providers are automatically collected, categorized and ranked in light of a user’s habits or interests. Moreover, our system can adapt itself towards a better performance. In order to develop such an adaptive system, we proposed a hybrid learning strategy; supervised learning is used to create an initial system configuration based on user’s feedbacks during registration, while an unsupervised learning scheme gradually updates the configuration by tracing the user’s behaviors as the system is being used. Simulation results demonstrate satisfactory performance.
6083596
Improved learning algorithms for mixture of experts in multiclass classification
Mixture of experts (ME) is a modular neural network architecture for supervised learning. A double-loop Expectation-Maximization (EM) algorithm has been introduced to the ME architecture for adjusting the parameters and the iteratively reweighted least squares (IRLS) algorithm is used to perform maximization in the inner loop [Jordan, M.I., Jacobs, R.A. (1994). Hierarchical mixture of experts and the EM algorithm, Neural Computation, 6(2), 181-214]. However, it is reported in literature that the IRLS algorithm is of instability and the ME architecture trained by the EM algorithm, where IRLS algorithm is used in the inner loop, often produces the poor performance in multiclass classification. In this paper, the reason of this instability is explored. We find out that due to an implicitly imposed incorrect assumption on parameter independence in multiclass classification, an incomplete Hessian matrix is used in that IRLS algorithm. Based on this finding, we apply the Newton-Raphson method to the inner loop of the EM algorithm in the case of multiclass classification, where the exact Hessian matrix is adopted. To tackle the expensive computation of the Hessian matrix and its inverse, we propose an approximation to the Newton-Raphson algorithm based on a so-called generalized Bernoulli density. The Newton-Raphson algorithm and its approximation have been applied to synthetic data, benchmark, and real-world multiclass classification tasks. For comparison, the IRLS algorithm and a quasi-Newton algorithm called BFGS have also been applied to the same tasks. Simulation results have shown that the use of the proposed learning algorithms avoids the instability problem and makes the ME architecture produce good performance in multiclass classification. In particular, our approximation algorithm leads to fast learning. In addition, the limitation of our approximation algorithm is also empirically investigated in this paper.
15381679
A Modular Neural Network Architecture for Pattern Classification Based on Different Feature Sets
We propose a novel connectionist method for the use of different feature sets in pattern classification. Unlike traditional methods, e.g., combination of multiple classifiers and use of a composite feature set, our method copes with the problem based on an idea of soft competition on different feature sets developed in our earlier work. An alternative modular neural network architecture is proposed to provide a more effective implementation of soft competition on different feature sets. The proposed architecture is interpreted as a generalized finite mixture model and, therefore, parameter estimation is treated as a maximum likelihood problem. An EM algorithm is derived for parameter estimation and, moreover, a model selection method is proposed to fit the proposed architecture to a specific problem. Comparative results are presented for the real world problem of speaker identification.
41845464
Advances in Natural Computation, First International Conference, ICNC 2005, Changsha, China, August 27-29, 2005, Proceedings, Part I
35419934
MATLAB Simulation of Gradient-Based Neural Network for Online Matrix Inversion
This paper investigates the simulation of a gradient-based recurrent neural network for online solution of the matrix-inverse problem. Several important techniques are employed as follows to simulate such a neural system. 1) Kronecker product of matrices is introduced to transform a matrix-differential-equation (MDE) to a vector-differential-equation (VDE); i.e., finally, a standard ordinary-differential-equation (ODE) is obtained. 2) MATLAB routine "ode45" is introduced to solve the transformed initial-value ODE problem. 3) In addition to various implementation errors, different kinds of activation functions are simulated to show the characteristics of such a neural network. Simulation results substantiate the theoretical analysis and efficacy of the gradient-based neural network for online constant matrix inversion.
15411856
Prediction of the Number of Helices for the Twilight Zone Proteins
Protein structure prediction is one of the core research areas in bioinformatics. This paper addresses the protein secondary structure prediction problem for the twilight zone proteins, which are characterized by low, about 25% homology to the sets of known sequences. The commonly used sequence alignment based algorithms fail to provide accurate prediction for sequences of such low homology, and thus alternative solutions should be sought. We propose a novel method that aims at the prediction of the number of helical structures based on the twilight zone protein sequences. The method is based on a custom designed and compact feature based sequences representation and applies a decision tree prediction algorithm. The performed experimental study shows superiority of the proposed method over three other prediction algorithms and the results provided by YASPIN algorithm, which is a state-of-the-art alignment based secondary structure prediction method designed using low homology sequences
41045084
Parallel algorithms of the Purcell method for direct solution of linear systems
In this paper, we first demonstrate that the classical Purcell's vector method when combined with row pivoting yields a consistently small growth factor in comparison to the well-known Gauss elimination method, the Gauss-Jordan method and the Gauss-Huard method with partial pivoting. We then present six parallel algorithms of the Purcell method that may be used for direct solution of linear systems. The algorithms differ in ways of pivoting and load balancing. We recommend algorithms V and VI for their reliability and algorithms III and IV for good load balance if local pivoting is acceptable. Some numerical results are presented.
1213875
A simplified LVI-based primal-dual neural network for repetitive motion planning of PA10 robot manipulator starting from different initial states
This paper presents a simplified primal-dual neural network based on linear variational inequalities (LVI) for online repetitive motion planning of PA10 robot manipulator. To do this, a drift-free criterion is exploited in the form of a quadratic function. In addition, the repetitive-motion-planning scheme could incorporate the joint limits and joint velocity limits simultaneously. Such a scheme is finally reformulated as a time-varying quadratic program (QP). As a QP real-time solver, the simplified LVI-based primal-dual neural network (LVI-PDNN) is designed based on the QP-LVI conversion and Karush-Kuhn-Tucker (KKT) conditions. It has a simple piecewise-linear dynamics and could globally exponentially converge to the optimal solution of strictly-convex quadratic-programs. The simplified LVI-PDNN model is simulated based on PA10 robot arm, and simulation results show the effective remedy of the joint angle drift problem of PA10 robot.
193129
Sequence based residue depth prediction using evolutionary information and predicted secondary structure
BackgroundResidue depth allows determining how deeply a given residue is buried, in contrast to the solvent accessibility that differentiates between buried and solvent-exposed residues. When compared with the solvent accessibility, the depth allows studying deep-level structures and functional sites, and formation of the protein folding nucleus. Accurate prediction of residue depth would provide valuable information for fold recognition, prediction of functional sites, and protein design.ResultsA new method, RDPred, for the real-value depth prediction from protein sequence is proposed. RDPred combines information extracted from the sequence, PSI-BLAST scoring matrices, and secondary structure predicted with PSIPRED. Three-fold/ten-fold cross validation based tests performed on three independent, low-identity datasets show that the distance based depth (computed using MSMS) predicted by RDPred is characterized by 0.67/0.67, 0.66/0.67, and 0.64/0.65 correlation with the actual depth, by the mean absolute errors equal 0.56/0.56, 0.61/0.60, and 0.58/0.57, and by the mean relative errors equal 17.0%/16.9%, 18.2%/18.1%, and 17.7%/17.6%, respectively. The mean absolute and the mean relative errors are shown to be statistically significantly better when compared with a method recently proposed by Yuan and Wang [Proteins 2008; 70:509–516]. The results show that three-fold cross validation underestimates the variability of the prediction quality when compared with the results based on the ten-fold cross validation. We also show that the hydrophilic and flexible residues are predicted more accurately than hydrophobic and rigid residues. Similarly, the charged residues that include Lys, Glu, Asp, and Arg are the most accurately predicted. Our analysis reveals that evolutionary information encoded using PSSM is characterized by stronger correlation with the depth for hydrophilic amino acids (AAs) and aliphatic AAs when compared with hydrophobic AAs and aromatic AAs. Finally, we show that the secondary structure of coils and strands is useful in depth prediction, in contrast to helices that have relatively uniform distribution over the protein depth. Application of the predicted residue depth to prediction of buried/exposed residues shows consistent improvements in detection rates of both buried and exposed residues when compared with the competing method. Finally, we contrasted the prediction performance among distance based (MSMS and DPX) and volume based (SADIC) depth definitions. We found that the distance based indices are harder to predict due to the more complex nature of the corresponding depth profiles.ConclusionThe proposed method, RDPred, provides statistically significantly better predictions of residue depth when compared with the competing method. The predicted depth can be used to provide improved prediction of both buried and exposed residues. The prediction of exposed residues has implications in characterization/prediction of interactions with ligands and other proteins, while the prediction of buried residues could be used in the context of folding predictions and simulations.