14078787
Maximum Margin Clustering Made Practical
Motivated by the success of large margin methods in supervised learning, maximum margin clustering (MMC) is a recent approach that aims at extending large margin methods to unsupervised learning. However, its optimization problem is nonconvex and existing MMC methods all rely on reformulating and relaxing the nonconvex optimization problem as semidefinite programs (SDP). Though SDP is convex and standard solvers are available, they are computationally very expensive and only small data sets can be handled. To make MMC more practical, we avoid SDP relaxations and propose in this paper an efficient approach that performs alternating optimization directly on the original nonconvex problem. A key step to avoid premature convergence in the resultant iterative procedure is to change the loss function from the hinge loss to the Laplacian/square loss so that overconfident predictions are penalized. Experiments on a number of synthetic and real-world data sets demonstrate that the proposed approach is more accurate, much faster (hundreds to tens of thousands of times faster), and can handle data sets that are hundreds of times larger than the largest data set reported in the MMC literature.
5858271
Prototype vector machine for large scale semi-supervised learning
Practical data mining rarely falls exactly into the supervised learning scenario. Rather, the growing amount of unlabeled data poses a big challenge to large-scale semi-supervised learning (SSL). We note that the computational intensiveness of graph-based SSL arises largely from the manifold or graph regularization, which in turn lead to large models that are difficult to handle. To alleviate this, we proposed the prototype vector machine (PVM), a highly scalable, graph-based algorithm for large-scale SSL. Our key innovation is the use of "prototypes vectors" for efficient approximation on both the graph-based regularizer and model representation. The choice of prototypes are grounded upon two important criteria: they not only perform effective low-rank approximation of the kernel matrix, but also span a model suffering the minimum information loss compared with the complete model. We demonstrate encouraging performance and appealing scaling properties of the PVM on a number of machine learning benchmark data sets.
5878411
Density-Weighted Nystrm Method for Computing Large Kernel Eigensystems
The Nystrm method is a well-known sampling-based technique for approximating the eigensystem of large kernel matrices. However, the chosen samples in the Nystrm method are all assumed to be of equal importance, which deviates from the integral equation that defines the kernel eigenfunctions. Motivated by this observation, we extend the Nystrm method to a more general, density-weighted version. We show that by introducing the probability density function as a natural weighting scheme, the approximation of the eigensystem can be greatly improved. An efficient algorithm is proposed to enforce such weighting in practice, which has the same complexity as the original Nystrm method and hence is notably cheaper than several other alternatives. Experiments on kernel principal component analysis, spectral clustering, and image segmentation demonstrate the encouraging performance of our algorithm.
7532408
Simplifying Mixture Models Through Function Approximation
The finite mixture model is widely used in various statistical learning problems. However, the model obtained may contain a large number of components, making it inefficient in practical applications. In this paper, we propose to simplify the mixture model by minimizing an upper bound of the approximation error between the original and the simplified model, under the use of the L 2 distance measure. This is achieved by first grouping similar components together and then performing local fitting through function approximation. The simplified model obtained can then be used as a replacement of the original model to speed up various algorithms involving mixture models during training (e.g., Bayesian filtering, belief propagation) and testing [e.g., kernel density estimation, support vector machine (SVM) testing]. Encouraging results are observed in the experiments on density estimation, clustering-based image segmentation, and simplification of SVM decision functions.
973036
Accelerated Convergence Using Dynamic Mean Shift
Mean shift is an iterative mode-seeking algorithm widely used in pattern recognition and computer vision. However, its convergence is sometimes too slow to be practical. In this paper, we improve the convergence speed of mean shift by dynamically updating the sample set during the iterations, and the resultant procedure is called dynamic mean shift (DMS). When the data is locally Gaussian, it can be shown that both the standard and dynamic mean shift algorithms converge to the same optimal solution. However, while standard mean shift only has linear convergence, the dynamic mean shift algorithm has superlinear convergence. Experiments on color image segmentation show that dynamic mean shift produces comparable results as the standard mean shift algorithm, but can significantly reduce the number of iterations for convergence and takes much less time.
4300632
Keyword Extraction Using Support Vector Machine
This paper is concerned with keyword extraction. By keyword extraction, we mean extracting a subset of words/phrases from a document that can describe the ‘meaning' of the document. Keywords are of benefit to many text mining applications. However, a large number of documents do not have keywords and thus it is necessary to assign keywords before enjoying the benefit from it. Several research efforts have been done on keyword extraction. These methods make use of the ‘global context information', which makes the performance of extraction restricted. A thorough and systematic investigation on the issue is thus needed. In this paper, we propose to make use of not only ‘global context information', but also ‘local context information' for extracting keywords from documents. As far as we know, utilizing both ‘global context information' and ‘local context information' in keyword extraction has not been sufficiently investigated previously. Methods for performing the tasks on the basis of Support Vector Machines have also been proposed in this paper. Features in the model have been defined. Experimental results indicate that the proposed SVM based method can significantly outperform the baseline methods for keyword extraction. The proposed method has been applied to document classification, a typical text mining processing. Experimental results show that the accuracy of document classification can be significantly improved by using the keyword extraction method.
1252619
A fast approximate algorithm for training L1-SVMs in primal space
We propose a novel and fast algorithm to train support vector machines (SVMs) in primal space, which solves an approximate optimization of SVMs with the properties of unconstraint, continuity and twice differentiability by utilizing the Newton optimization technique. Further, we devise a special pre-extracting procedure to speed up the convergence of the algorithm by resorting to a high-quality initial solution. Theoretical studies show that the proposed algorithm produces an @?-approximate solution to standard SVMs and maintains low computational complexity. Experimental results on benchmark data sets demonstrate that our algorithm is much faster than the dual based method such as SVM^l^i^g^h^t while it achieves the similar test accuracy.
207113548
A Hybrid Two-Phase Algorithm For Face Recognition
Scientists have developed numerous classifiers in the pattern recognition field, because applying a single classifier is not very conducive to achieve a high recognition rate on face databases. Problems occur when the images of the same person are classified as one class, while they are in fact different in poses, expressions, or lighting conditions. In this paper, we present a hybrid, two-phase face recognition algorithm to achieve high recognition rates on the FERET data set. The first phase is to compress the large class number database size, whereas the second phase is to perform the decision-making. We investigate a variety of combinations of the feature extraction and pattern classification methods. Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and Support Vector Machine (SVM) are examined and tested using 700 facial images of different poses from FERET database. Experimental results show that the two combinations, LDA+LDA and LDA+SVM, outperform the other types of combinations. Meanwhile, when classifiers are considered in the two-phase face recognition, it is better to adopt the L1 distance in the first phase and the class mean in the second phase.
32209678
A Novel Coarse-to-Fine Adaptation Segmentation Approach for Cellular Image Analysis
Cellular image content analysis is one of the most important aspects of the cellular research and often requires collecting a great amount of statistical information and phenomena. Automated segmentation of time-lapse images gradually becomes the key problem in cellular image analysis. To address fuzzy, irregular, and ruffling cell boundaries in time-lapse cellular images, this paper introduces a hierarchical coarse-to-fine approach which is composed of iteration-dependent adaptation procedures with high-level interpretation: initial segmentation, adaptive processing, and refined segmentation. The iteration-dependent adaptation lies in that the adaptive processing and the refined segmentation be deliberately designed without a fixed order and a uniform associated iteration number, to connect coarse segmentation and refined segmentation for locally progressive approximation. The initial segmentation could avoid over-segmentation from watershed transform and converge to some features using a priori information. Experimental results on cellular images with spurious branches, arbitrary gaps, low contrast boundaries and low signal-to-noise ratio, show that the proposed approach provides a close matching to the manual cognition and overcomes several common drawbacks from other existing methods applied on cell migration. The procedure configuration of the proposed approach has a certain potential to serve as a biomedical image content analysis tool.
16445417
Multiple Viewpoints Based Ontology Integration
Ontology integration is a focus on ontology application field. Ontology can be viewed as a kind of software product. Ontology integration needs to be directed by methodology. In many applications, we need to integrate existed ontologies for a unified ontology for application requirements. The ontology to be integrated can be viewed as a viewpoint of the unified ontology. A multiple-viewpoints-based ontology integration approach is introduced by multiple viewpoints theory in requirement engineering. We define ontology viewpoint by characters of ontology and use conceptual graph to represent semantic in ontology. We discuss the inconsistency checking inner ontology viewpoint and among ontology viewpoints. At last, we use a concept lattice to construct the concept hierarchy.