9123990
Dimension Reduction in Text Classification with Support Vector Machines
Support vector machines (SVMs) have been recognized as one of the most successful classification methods for many applications including text classification. Even though the learning ability and computational complexity of training in support vector machines may be independent of the dimension of the feature space, reducing computational complexity is an essential issue to efficiently handle a large number of terms in practical applications of text classification. In this paper, we adopt novel dimension reduction methods to reduce the dimension of the document vectors dramatically. We also introduce decision functions for the centroid-based classification algorithm and support vector classifiers to handle the classification problem where a document may belong to multiple classes. Our substantial experimental results show that with several dimension reduction methods that are designed particularly for clustered data, higher efficiency for both training and testing can be achieved without sacrificing prediction accuracy of text classification even when the dimension of the input space is significantly reduced.
11702143
Adaptive nonlinear discriminant analysis by regularized minimum squared errors
Kernelized nonlinear extensions of Fisher's discriminant analysis, discriminant analysis based on generalized singular value decomposition (LDA/GSVD), and discriminant analysis based on the minimum squared error formulation (MSE) have recently been widely utilized for handling undersampled high-dimensional problems and nonlinearly separable data sets. As the data sets are modified from incorporating new data points and deleting obsolete data points, there is a need to develop efficient updating and downdating algorithms for these methods to avoid expensive recomputation of the solution from scratch. In this paper, an efficient algorithm for adaptive linear and nonlinear kernel discriminant analysis based on regularized MSE, called adaptive KDA/RMSE, is proposed. In adaptive KDA/RMSE, updating and downdating of the computationally expensive eigenvalue decomposition (EVD) or singular value decomposition (SVD) is approximated by updating and downdating of the QR decomposition achieving an order of magnitude speed up. This fast algorithm for adaptive kernelized discriminant analysis is designed by utilizing regularization techniques and the relationship between linear and nonlinear discriminant analysis and the MSE. In addition, an efficient algorithm to compute leave-one-out cross validation is also introduced by utilizing downdating of KDA/RMSE.
14943272
Incremental and Decremental Least Squares Support Vector Machine and Its Application to Drug Design
The least squares support vector machine (LS-SVM) has shown to exhibit excellent classification performance in many applications. In this paper, we propose an incremental and decremental LS-SVM based on updating and downdating the QR decomposition. It can efficiently compute the updated solution when data points are appended or removed. The experiment results illustrated that the proposed incremental algorithm efficiently produces the same solutions as those obtained by LS-SVM which recomputes the solution all over even for small changes in the data. For drug design, the results of each biochemistry laboratory test on a new compound can be iteratively included in the training set. This procedure can further improve precision in order to select the next best predicted organic compound. Instead of retraining entire data points, it is much efficient to update solution by incremental LS-SVM.
5469426
Unraveling condition specific gene transcriptional regulatory networks in Saccharomyces cerevisiae
BackgroundGene expression and transcription factor (TF) binding data have been used to reveal gene transcriptional regulatory networks. Existing knowledge of gene regulation can be presented using gene connectivity networks. However, these composite connectivity networks do not specify the range of biological conditions of the activity of each link in the network.ResultsWe present a novel method that utilizes the expression and binding patterns of the neighboring nodes of each link in existing experimentally-based, literature-derived gene transcriptional regulatory networks and extend them in silico using TF-gene binding motifs and a compendium of large expression data from Saccharomyces cerevisiae. Using this method, we predict several hundreds of new transcriptional regulatory TF-gene links, along with experimental conditions in which known and predicted links become active. This approach unravels new links in the yeast gene transcriptional regulatory network by utilizing the known transcriptional regulatory interactions, and is particularly useful for breaking down the composite transcriptional regulatory network to condition specific networks.ConclusionOur methods can facilitate future binding experiments, as they can considerably help focus on the TFs that must be surveyed to understand gene regulation.(Supplemental material and the latest version of the MATLAB implementation of the United Signature Algorithm is available online at [1] or [see Additional files 1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
1329357
Data Reduction in Support Vector Machines by a Kernelized Ionic Interaction Model
A major drawback of support vector machines is their high computational complexity. In this paper, we introduce a novel kernelized ionic interaction (IoI) model for data reduction in support vector machines. We also present a data reduction method based on the kernelized instance based (KIB2) algorithm. We show that the computation time can be significantly reduced without any significant decrease in the prediction accuracy.
8166541
Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis
MOTIVATION
Many practical pattern recognition problems require non-negativity constraints. For example, pixels in digital images and chemical concentrations in bioinformatics are non-negative. Sparse non-negative matrix factorizations (NMFs) are useful when the degree of sparseness in the non-negative basis matrix or the non-negative coefficient matrix in an NMF needs to be controlled in approximating high-dimensional data in a lower dimensional space.


RESULTS
In this article, we introduce a novel formulation of sparse NMF and show how the new formulation leads to a convergent sparse NMF algorithm via alternating non-negativity-constrained least squares. We apply our sparse NMF algorithm to cancer-class discovery and gene expression data analysis and offer biological analysis of the results obtained. Our experimental results illustrate that the proposed sparse NMF algorithm often achieves better clustering performance with shorter computing time compared to other existing NMF algorithms.


AVAILABILITY
The software is available as supplementary material.
2872153
2D-profiling: detecting input-dependent branches with a single input data set
Static compilers use profiling to predict run-time program behavior. Generally, this requires multiple input sets to capture wide variations in run-time behavior. This is expensive in terms of resources and compilation time. We introduce a new mechanism, 2D-profiling, which profiles with only one input set and predicts whether the result of the profile would change significantly across multiple input sets. We use 2D-profiling to predict whether a branch's prediction accuracy varies across input sets. The key insight is that if the prediction accuracy of an individual branch varies significantly over a profiling run with one input set, then it is more likely that the prediction accuracy of that branch varies across input sets. We evaluate 2D-profiling with the SPEC CPU 2000 integer benchmarks and show that it can identify input-dependent branches accurately.
36946020
Face Representation Method Using Pixel-to-Vertex Map (PVM) for 3D Model Based Face Recognition
3D model based approach for face recognition has been spotlighted as a robust solution under variant conditions of pose and illumination. Since a generative 3D face model consists of a large number of vertices, a 3D model based face recognition system is generally inefficient in computation time. In this paper, we propose a novel 3D face representation algorithm to reduce the number of vertices and optimize its computation time. Finally, we evaluate the performance of proposed algorithm with the Korean face database collected using a stereo-camera based 3D face capturing device.
4665838
Independent component analysis in a facial local residue space
In this paper, we propose an ICA (Independent Component Analysis) based face recognition algorithm, which is robust to illumination and pose variation. Generally, it is well known that the first few eigenfaces represent illumination variation rather than identity. Most PCA (Principal Component Analysis)-based methods have overcome illumination variation by discarding the projection to a few leading eigenfaces. The space spanned after removing a few leading eigenfaces is called the "residual face space". We found that ICA in the residual face space provides more efficient encoding in terms of redundancy reduction and robustness to pose variation as well as illumination variation, owing to its ability to represent non-Gaussian statistics. Moreover, a face image is separated into several facial components, local spaces, and each local space is represented by the ICA bases (independent components) of its corresponding residual space. The statistical models of face images in local spaces are relatively simple and facilitate classification by a linear encoding. Various experimental results show that the accuracy of face recognition is significantly improved by the proposed method under large illumination and pose variations.
4644097
Component-based LDA face description for image retrieval and MPEG-7 standardisation
We propose a method of face description for facial image retrieval from a large data base and for MPEG-7 (Moving Picture Experts Group) standardisation. The novel descriptor is obtained by decomposing a face image into several components and then combining the component features. The decomposition combined with LDA (Linear Discriminant Analysis) provides discriminative facial descriptions that are less sensitive to light and pose changes. Each facial component is represented in its Fisher space and another LDA is then applied to compactly combine the features of the components. To enhance retrieval accuracy further, a simple pose classification and transformation technique is performed, followed by recursive matching. Our algorithm has been developed to deal with the problem of face image retrieval from huge databases such as those found in Internet environments. Such retrieval requires a compact face representation which has robust recognition performance under lighting and pose variations. The partitioning of a face image into components offers a number of benefits that facilitate the development of an efficient and robust face retrieval algorithm. Variation in image statistics due to pose and/or illumination changes within each component region can be simplified and more easily captured by a linear encoding than that of the whole image. So an LDA encoding at the component level facilitates better classification. Furthermore, a facial component can be weighted according to its importance. The component with a large variation is weighted less in the matching stage to yield a more reliable decision. The experimental results obtained on the MPEG-7 data set show an impressive accuracy of our algorithm as compared with other methods including conventional PCA (Principal Component Analysis)/ICA (Independent Component Analysis)/LDA methods and the previous MPEG-7 proposals.
5973251
Secure authentication watermarking for binary images
Authentication watermarking is a hidden data inserted into an image, in order to detect any alterations. It seems to be almost impossible to design a really secure authentication watermarking without making use of the solid cryptography theory and techniques. In a cryptography-based authentication watermarking, a message authentication code (or digital signature) of the whole image is computed and the resulting code is inserted into the image itself. However, inserting the code alters the image and consequently its authentication code, invalidating the watermark. To avoid this problem, for gray-scale or color image, usually the least significant bits (LSBs) are cleared, the authentication code of the LSB-cleared image is computed and then the code is inserted into LSBs. Surely, one cannot perform the same procedure for binary images. We propose a quite simple solution for inserting a secure authentication watermarking in dispersed-dot halftone images. This technique can also be applied to any kind of binary images (including clustered-dot halftones), though the visual quality is not as good as when applied to dispersed-dot halftones. The proposed technique can be used with both secret-key or public-key ciphers.