1757195
Combining linear discriminant functions with neural networks for supervised learning
A novel supervised learning method is proposed by combining linear discriminant functions with neural networks. The proposed method results in a tree-structured hybrid architecture. Due to constructive learning, the binary tree hierarchical architecture is automatically generated by a controlled growing process for a specific supervised learning task. Unlike the classic decision tree, the linear discriminant functions are merely employed in the intermediate level of the tree for heuristically partitioning a large and complicated task into several smaller and simpler subtasks in the proposed method. These subtasks are dealt with by component neural networks at the leaves of the tree accordingly. For constructive learning, growing and credit-assignment algorithms are developed to serve for the hybrid architecture. The proposed architecture provides an efficient way to apply existing neural networks (e.g. multi-layered perceptron) for solving a large scale problem. We have already applied the proposed method to a universal approximation problem and several benchmark classification problems in order to evaluate its performance. Simulation results have shown that the proposed method yields better results and faster training in comparison with the multilayered perceptron.
15908834
Ensemble Learning with Active Data Selection for Semi-Supervised Pattern Classification
Unlike traditional pattern classification, semi-supervised learning provides a novel technique to make use of both labeled and unlabeled data for improving the performance of classification. In general, there are two critical issues for semi-supervised learning of discriminative classifiers; i.e., how to create an initial classifier of a good generalization capability with the limited labeled data and the how to make an effective use of unlabeled data without degradation of the established classifier. To tackle two aforementioned problems, we propose an ensemble learning approach based on a recent active data selection strategy, where ensemble learning would yield good generalization and active data selection tends to choose the unlabeled data more likely resulting in an improvement during semi-supervised learning. By using an ensemble of K-NN classifiers, we demonstrate the effectiveness of our approach on a synthetic data classification and a facial expression recognition tasks.
41845464
Advances in Natural Computation, First International Conference, ICNC 2005, Changsha, China, August 27-29, 2005, Proceedings, Part I
15381679
A Modular Neural Network Architecture for Pattern Classification Based on Different Feature Sets
We propose a novel connectionist method for the use of different feature sets in pattern classification. Unlike traditional methods, e.g., combination of multiple classifiers and use of a composite feature set, our method copes with the problem based on an idea of soft competition on different feature sets developed in our earlier work. An alternative modular neural network architecture is proposed to provide a more effective implementation of soft competition on different feature sets. The proposed architecture is interpreted as a generalized finite mixture model and, therefore, parameter estimation is treated as a maximum likelihood problem. An EM algorithm is derived for parameter estimation and, moreover, a model selection method is proposed to fit the proposed architecture to a specific problem. Comparative results are presented for the real world problem of speaker identification.
6083596
Improved learning algorithms for mixture of experts in multiclass classification
Mixture of experts (ME) is a modular neural network architecture for supervised learning. A double-loop Expectation-Maximization (EM) algorithm has been introduced to the ME architecture for adjusting the parameters and the iteratively reweighted least squares (IRLS) algorithm is used to perform maximization in the inner loop [Jordan, M.I., Jacobs, R.A. (1994). Hierarchical mixture of experts and the EM algorithm, Neural Computation, 6(2), 181-214]. However, it is reported in literature that the IRLS algorithm is of instability and the ME architecture trained by the EM algorithm, where IRLS algorithm is used in the inner loop, often produces the poor performance in multiclass classification. In this paper, the reason of this instability is explored. We find out that due to an implicitly imposed incorrect assumption on parameter independence in multiclass classification, an incomplete Hessian matrix is used in that IRLS algorithm. Based on this finding, we apply the Newton-Raphson method to the inner loop of the EM algorithm in the case of multiclass classification, where the exact Hessian matrix is adopted. To tackle the expensive computation of the Hessian matrix and its inverse, we propose an approximation to the Newton-Raphson algorithm based on a so-called generalized Bernoulli density. The Newton-Raphson algorithm and its approximation have been applied to synthetic data, benchmark, and real-world multiclass classification tasks. For comparison, the IRLS algorithm and a quasi-Newton algorithm called BFGS have also been applied to the same tasks. Simulation results have shown that the use of the proposed learning algorithms avoids the instability problem and makes the ME architecture produce good performance in multiclass classification. In particular, our approximation algorithm leads to fast learning. In addition, the limitation of our approximation algorithm is also empirically investigated in this paper.
34883067
Personalized News Reading via Hybrid Learning
In this paper, we present a personalized news reading prototype where latest news articles published by various on-line news providers are automatically collected, categorized and ranked in light of a user’s habits or interests. Moreover, our system can adapt itself towards a better performance. In order to develop such an adaptive system, we proposed a hybrid learning strategy; supervised learning is used to create an initial system configuration based on user’s feedbacks during registration, while an unsupervised learning scheme gradually updates the configuration by tracing the user’s behaviors as the system is being used. Simulation results demonstrate satisfactory performance.
13931461
Accurate sequence-based prediction of catalytic residues
MOTIVATION
Prediction of catalytic residues provides useful information for the research on function of enzymes. Most of the existing prediction methods are based on structural information, which limits their use. We propose a sequence-based catalytic residue predictor that provides predictions with quality comparable to modern structure-based methods and that exceeds quality of state-of-the-art sequence-based methods.


RESULTS
Our method (CRpred) uses sequence-based features and the sequence-derived PSI-BLAST profile. We used feature selection to reduce the dimensionality of the input (and explain the input) to support vector machine (SVM) classifier that provides predictions. Tests on eight datasets and side-by-side comparison with six modern structure- and sequence-based predictors show that CRpred provides predictions with quality comparable to current structure-based methods and better than sequence-based methods. The proposed method obtains 15-19% precision and 48-58% TP (true positive) rate, depending on the dataset used. CRpred also provides confidence values that allow selecting a subset of predictions with higher precision. The improved quality is due to newly designed features and careful parameterization of the SVM. The features incorporate amino acids characterized by the highest and the lowest propensities to constitute catalytic residues, Gly that provides flexibility for catalytic sites and sequence motifs characteristic to certain catalytic reactions. Our features indicate that catalytic residues are on average more conserved when compared with the general population of residues and that highly conserved amino acids characterized by high catalytic propensity are likely to form catalytic sites. We also show that local (with respect to the sequence) hydrophobicity contributes towards the prediction.
1589839
Optimization of the Sliding Window Size for Protein Structure Prediction
Sliding window based methods are relatively often applied in prediction of various aspects related to protein structure. Despite their wide spread use, researchers did not establish a standard related to the size of the window, i.e., window sizes ranging between 7 and 17 residues were used in the past. To this end, this paper performs a computational study based on a probabilistic approach that aims at finding an optimal sliding window size. The results shows that formation of helical structure can be affected by amino acids (AAs) that are up to 9 positions away in the sequence, while the formation of coils and strands can be affected by AAs that are up to 3 and 6 positions away, respectively. Overall, our results suggest that a sliding window with 19 residues is optimal for secondary structure prediction, while for a specific prediction tasks, such as prediction of p-strands, a smaller window size is sufficient. Finally, the 20 AAs are categorized into five groups based on their influence of formation of the secondary structure. The finding related to the optimal window size was confirmed based on an independent experimental study related to the prediction of secondary protein structure
15894335
An adaptive load balancing framework for parallel database systems based on collaborative agents
Load balancing is a key technique in parallel computer supported collaborative work (CSCW) systems, parallel database system and P2P system for instance, to boost performance and improve scalability. In order to reduce total cost of ownership (TCO), adaptive/self-tuning administration techniques are gradually and extensively expected in the cyberspace. In parallel database systems, adaptive load balancing techniques are proposed to face the change in data storage patterns and access patterns in a dynamic real environment. The techniques utilized in both shared-nothing and shared-disk parallel database systems are discussed, and a general flexible framework based on collaborative agents is studied to support these techniques in both architectures. The framework supports two kinds of load balancing - one is passively executing query statements balancedly, and the other one is proactively adjusting data placement and task execution scheme, by means of data and task migration, whenever load unbalance is detected. Three categories of agents, scheduling agents, monitoring agents and task agents, are identified in the framework. The collaboration protocols and scheduling algorithms to support adaptive load balancing are described. The framework also applies to other parallel systems such as P2P systems and shared file processing systems due to their underlying commonness.
6117831
MATLAB Simulink modeling and simulation of LVI-based primal-dual neural network for solving linear and quadratic programs
In view of parallel-processing nature and circuit-implementation convenience, recurrent neural networks are often employed to solve optimization problems. Recently, a primal-dual neural network based on linear variational inequalities (LVI) was developed by Zhang et al. for the online solution of linear-programming (LP) and quadratic-programming (QP) problems simultaneously subject to equality, inequality and bound constraints. For the final purpose of field programmable gate array (FPGA) and application-specific integrated circuit (ASIC) realization, we investigate in this paper the MATLAB Simulink modeling and simulative verification of such an LVI-based primal-dual neural network (LVI-PDNN). By using click-and-drag mouse operations in MATLAB Simulink environment, we could quickly model and simulate complicated dynamic systems. Modeling and simulative results substantiate the theoretical analysis and efficacy of the LVI-PDNN for solving online the linear and quadratic programs.
3106142
MATLAB Simulation and Comparison of Zhang Neural Network and Gradient Neural Network for Online Solution of Linear Time-Varying Matrix Equation AXB-C=0
Different from gradient neural networks (GNN), a special kind ofrecurrent neural networks has been proposed recently by Zhanget alfor solving online linear matrix equations withtime-varying coefficients. Such recurrent neural networks, designedbased on a matrix-valued error-function, could achieve globalexponential convergence when solving online time-varying problemsin comparison with gradient neural networks. This paperinvestigates the MATLAB simulation of Zhang neural networks (ZNN)for real-time solution of linear time-varying matrix equationAXB- C= 0. Gradient neural networks aresimulated and compared as well. Simulation results substantiate thetheoretical analysis and efficacy of ZNN on linear time-varyingmatrix equation solving.