7532408
Simplifying Mixture Models Through Function Approximation
The finite mixture model is widely used in various statistical learning problems. However, the model obtained may contain a large number of components, making it inefficient in practical applications. In this paper, we propose to simplify the mixture model by minimizing an upper bound of the approximation error between the original and the simplified model, under the use of the L 2 distance measure. This is achieved by first grouping similar components together and then performing local fitting through function approximation. The simplified model obtained can then be used as a replacement of the original model to speed up various algorithms involving mixture models during training (e.g., Bayesian filtering, belief propagation) and testing [e.g., kernel density estimation, support vector machine (SVM) testing]. Encouraging results are observed in the experiments on density estimation, clustering-based image segmentation, and simplification of SVM decision functions.
1966273
Block-quantized kernel matrix for fast spectral embedding
Eigendecomposition of kernel matrix is an indispensable procedure in many learning and vision tasks. However, the cubic complexity <i>O</i>(<i>N</i><sup>3</sup>) is impractical for large problem, where <i>N</i> is the data size. In this paper, we propose an efficient approach to solve the eigendecomposition of the kernel matrix <i>W</i>. The idea is to approximate <i>W</i> with <i>W</i> that is composed of <i>m</i><sup>2</sup> constant blocks. The eigenvectors of <i>W</i>, which can be solved in <i>O</i>(<i>m</i><sup>3</sup>) time, is then used to recover the eigenvectors of the original kernel matrix. The complexity of our method is only <i>O</i>(<i>mN</i> + <i>m</i><sup>3</sup>), which scales more favorably than state-of-the-art low rank approximation and sampling based approaches (<i>O</i>(<i>m</i><sup>2</sup><i>N</i> + <i>m</i><sup>3</sup>)), and the approximation quality can be controlled conveniently. Our method demonstrates encouraging scaling behaviors in experiments of image segmentation (by spectral clustering) and kernel principal component analysis.
5858271
Prototype vector machine for large scale semi-supervised learning
Practical data mining rarely falls exactly into the supervised learning scenario. Rather, the growing amount of unlabeled data poses a big challenge to large-scale semi-supervised learning (SSL). We note that the computational intensiveness of graph-based SSL arises largely from the manifold or graph regularization, which in turn lead to large models that are difficult to handle. To alleviate this, we proposed the prototype vector machine (PVM), a highly scalable, graph-based algorithm for large-scale SSL. Our key innovation is the use of "prototypes vectors" for efficient approximation on both the graph-based regularizer and model representation. The choice of prototypes are grounded upon two important criteria: they not only perform effective low-rank approximation of the kernel matrix, but also span a model suffering the minimum information loss compared with the complete model. We demonstrate encouraging performance and appealing scaling properties of the PVM on a number of machine learning benchmark data sets.
6920854
Chaos inducement and Enhancement in Two Particular Nonlinear Maps Using Weak Periodic/quasiperiodic Perturbations
Weak periodic perturbation has long been used to suppress chaos in dynamical systems. In this paper, however, we demonstrate that weak periodic or quasiperiodic perturbation can also be used to induce chaos in nonchaotic parameter ranges of chaotic maps, or to enhance the already existing chaotic state. Two kinds of chaotic maps, the period doubling system and the Hopf bifurcation system, are employed as basic models to analyze and compare in detail the different mechanisms of inducing and enhancing chaos in them. In addition, a special kind of intermittency characterized by its periodicity is found for the first time in periodically perturbed Henon map, and reasonable speculations are presented to explain its complicated dynamics.
10651609
Improved Nyström low-rank approximation and error analysis
Low-rank matrix approximation is an effective tool in alleviating the memory and computational burdens of kernel methods and sampling, as the mainstream of such algorithms, has drawn considerable attention in both theory and practice. This paper presents detailed studies on the Nyström sampling scheme and in particular, an error analysis that directly relates the Nyström approximation quality with the encoding powers of the landmark points in summarizing the data. The resultant error bound suggests a simple and efficient sampling scheme, the k-means clustering algorithm, for Nyström low-rank approximation. We compare it with state-of-the-art approaches that range from greedy schemes to probabilistic sampling. Our algorithm achieves significant performance gains in a number of supervised/unsupervised learning tasks including kernel PCA and least squares SVM.
2257586
Applying neighborhood consistency for fast clustering and kernel density estimation
Nearest neighborhood consistency is an important concept in statistical pattern recognition, which underlies the well-known k-nearest neighbor method. In this paper, we combine this idea with kernel density estimation based clustering, and derive the fast mean shift algorithm (FMS). FMS greatly reduces the complexity of feature space analysis, resulting satisfactory precision of classification. More importantly, we show that with FMS algorithm, we are in fact relying on a conceptually novel approach of density estimation, the fast kernel density estimation (FKDE) for clustering. The FKDE combines smooth and non-smooth estimators and thus inherits advantages from both. Asymptotic analysis reveals the approximation of the FKDE to standard kernel density estimator. Data clustering and image segmentation experiments demonstrate the efficiency of FMS.
17724018
Logical structure based semantic relationship extraction from semi-structured documents
Addressed in this paper is the issue of semantic relationship extraction from semi-structured documents. Many research efforts have been made so far on the semantic information extraction. However, much of the previous work focuses on detecting `isolated' semantic information by making use of linguistic analysis or linkage information in web pages and limited research has been done on extracting semantic relationship from the semi-structured documents. In this paper, we propose a method for semantic relationship extraction by using the logical information in the semi-structured document (semi-structured document usually has various types of structure information, e.g. a semi-structured document may be hierarchical laid out). To the best of our knowledge, extracting semantic relationships by using logical information has not been investigated previously. A probabilistic approach has been proposed in the paper. Features used in the probabilistic model have been defined.
10595235
A Block Coprocessor for User Data Rate Improvements to GPRS Coding Scheme 4
The general packet radio service (GPRS) has been developed to allow packet data to be transported efficiently over an existing circuit-switched radio network, such as GSM. The main application of GPRS are in transporting Internet protocol (IP) datagrams from web servers (for telemetry or for mobile Internet browsers). Four GPRS baseband coding schemes are defined to offer a trade-off in requested data rates versus propagation channel conditions. However, data rates in the order of > 100 kbits/s are only achievable if the simplest coding scheme is used (CS-4) which offers little error detection and correction (EDC) (requiring excellent SNR) and the receiver hardware is capable of full duplex which is not currently available in the consumer market. A simple EDC scheme to improve the GPRS block error rate (BLER) performance is presented, particularly for CS-4, however gains in other coding schemes are seen. For every GPRS radio block that is corrected by the EDC scheme, the block does not need to be retransmitted releasing bandwidth in the channel and improving the user's application data rate. As GPRS requires intensive processing in the baseband, a viable field programmable gate array (FPGA) solution is presented in this paper.
4300632
Keyword Extraction Using Support Vector Machine
This paper is concerned with keyword extraction. By keyword extraction, we mean extracting a subset of words/phrases from a document that can describe the ‘meaning' of the document. Keywords are of benefit to many text mining applications. However, a large number of documents do not have keywords and thus it is necessary to assign keywords before enjoying the benefit from it. Several research efforts have been done on keyword extraction. These methods make use of the ‘global context information', which makes the performance of extraction restricted. A thorough and systematic investigation on the issue is thus needed. In this paper, we propose to make use of not only ‘global context information', but also ‘local context information' for extracting keywords from documents. As far as we know, utilizing both ‘global context information' and ‘local context information' in keyword extraction has not been sufficiently investigated previously. Methods for performing the tasks on the basis of Support Vector Machines have also been proposed in this paper. Features in the model have been defined. Experimental results indicate that the proposed SVM based method can significantly outperform the baseline methods for keyword extraction. The proposed method has been applied to document classification, a typical text mining processing. Experimental results show that the accuracy of document classification can be significantly improved by using the keyword extraction method.
28737704
Feature-Correlation Based Multi-view Detection
A view validation algorithm has been shown to predict whether or not the views are sufficiently compatible for solving a particular learning task. But it only works when a natural split of features exists. If the split does not exist, it will fail to manufacture a feature split to build the best views. In this paper, we present a general algorithm CCFP (Correlation and Compatibility based Feature Partitioner) to automate multi-view detection. CCFP first labels the large amount of unlabeled examples using single view algorithm, then calculates the conditional SU (Symmetric Uncertainty) between every pair of features and the IG (Information Gain) of each feature given the examples labeled previously by single view algorithm with high-confidence predictions. According to the estimated values of SU and IG, all the features will be partitioned into two views that are low correlated, compatible and sufficient enough. The experiment results show that multi-view learner with views generated by CCFP outperforms learner with views generated by other means clearly.
44494763
Inter-Frame Interpolation By Snake Model And Greedy Algorithm
In this paper, we present a novel method to solve the inter-frame interpolation problem in image morphing. We use our improved snake model that is associated with the gravitational force field to locate control points in the object contours. Afterwards, we apply the greedy algorithm in free-form deformations to achieve optimal warps among feature point pairs in starting and ending frames. The new method uses an energy-minimization function under the influence of inter-frames. The energy serves to impose frame-wise and curve-wise constraints among the interpolated frames.