1966273
Block-quantized kernel matrix for fast spectral embedding
Eigendecomposition of kernel matrix is an indispensable procedure in many learning and vision tasks. However, the cubic complexity <i>O</i>(<i>N</i><sup>3</sup>) is impractical for large problem, where <i>N</i> is the data size. In this paper, we propose an efficient approach to solve the eigendecomposition of the kernel matrix <i>W</i>. The idea is to approximate <i>W</i> with <i>W</i> that is composed of <i>m</i><sup>2</sup> constant blocks. The eigenvectors of <i>W</i>, which can be solved in <i>O</i>(<i>m</i><sup>3</sup>) time, is then used to recover the eigenvectors of the original kernel matrix. The complexity of our method is only <i>O</i>(<i>mN</i> + <i>m</i><sup>3</sup>), which scales more favorably than state-of-the-art low rank approximation and sampling based approaches (<i>O</i>(<i>m</i><sup>2</sup><i>N</i> + <i>m</i><sup>3</sup>)), and the approximation quality can be controlled conveniently. Our method demonstrates encouraging scaling behaviors in experiments of image segmentation (by spectral clustering) and kernel principal component analysis.
14078787
Maximum Margin Clustering Made Practical
Motivated by the success of large margin methods in supervised learning, maximum margin clustering (MMC) is a recent approach that aims at extending large margin methods to unsupervised learning. However, its optimization problem is nonconvex and existing MMC methods all rely on reformulating and relaxing the nonconvex optimization problem as semidefinite programs (SDP). Though SDP is convex and standard solvers are available, they are computationally very expensive and only small data sets can be handled. To make MMC more practical, we avoid SDP relaxations and propose in this paper an efficient approach that performs alternating optimization directly on the original nonconvex problem. A key step to avoid premature convergence in the resultant iterative procedure is to change the loss function from the hinge loss to the Laplacian/square loss so that overconfident predictions are penalized. Experiments on a number of synthetic and real-world data sets demonstrate that the proposed approach is more accurate, much faster (hundreds to tens of thousands of times faster), and can handle data sets that are hundreds of times larger than the largest data set reported in the MMC literature.
5878411
Density-Weighted Nystrm Method for Computing Large Kernel Eigensystems
The Nystrm method is a well-known sampling-based technique for approximating the eigensystem of large kernel matrices. However, the chosen samples in the Nystrm method are all assumed to be of equal importance, which deviates from the integral equation that defines the kernel eigenfunctions. Motivated by this observation, we extend the Nystrm method to a more general, density-weighted version. We show that by introducing the probability density function as a natural weighting scheme, the approximation of the eigensystem can be greatly improved. An efficient algorithm is proposed to enforce such weighting in practice, which has the same complexity as the original Nystrm method and hence is notably cheaper than several other alternatives. Experiments on kernel principal component analysis, spectral clustering, and image segmentation demonstrate the encouraging performance of our algorithm.
5858271
Prototype vector machine for large scale semi-supervised learning
Practical data mining rarely falls exactly into the supervised learning scenario. Rather, the growing amount of unlabeled data poses a big challenge to large-scale semi-supervised learning (SSL). We note that the computational intensiveness of graph-based SSL arises largely from the manifold or graph regularization, which in turn lead to large models that are difficult to handle. To alleviate this, we proposed the prototype vector machine (PVM), a highly scalable, graph-based algorithm for large-scale SSL. Our key innovation is the use of "prototypes vectors" for efficient approximation on both the graph-based regularizer and model representation. The choice of prototypes are grounded upon two important criteria: they not only perform effective low-rank approximation of the kernel matrix, but also span a model suffering the minimum information loss compared with the complete model. We demonstrate encouraging performance and appealing scaling properties of the PVM on a number of machine learning benchmark data sets.
10651609
Improved Nyström low-rank approximation and error analysis
Low-rank matrix approximation is an effective tool in alleviating the memory and computational burdens of kernel methods and sampling, as the mainstream of such algorithms, has drawn considerable attention in both theory and practice. This paper presents detailed studies on the Nyström sampling scheme and in particular, an error analysis that directly relates the Nyström approximation quality with the encoding powers of the landmark points in summarizing the data. The resultant error bound suggests a simple and efficient sampling scheme, the k-means clustering algorithm, for Nyström low-rank approximation. We compare it with state-of-the-art approaches that range from greedy schemes to probabilistic sampling. Our algorithm achieves significant performance gains in a number of supervised/unsupervised learning tasks including kernel PCA and least squares SVM.
1252619
A fast approximate algorithm for training L1-SVMs in primal space
We propose a novel and fast algorithm to train support vector machines (SVMs) in primal space, which solves an approximate optimization of SVMs with the properties of unconstraint, continuity and twice differentiability by utilizing the Newton optimization technique. Further, we devise a special pre-extracting procedure to speed up the convergence of the algorithm by resorting to a high-quality initial solution. Theoretical studies show that the proposed algorithm produces an @?-approximate solution to standard SVMs and maintains low computational complexity. Experimental results on benchmark data sets demonstrate that our algorithm is much faster than the dual based method such as SVM^l^i^g^h^t while it achieves the similar test accuracy.
15012237
A topical PageRank based algorithm for recommender systems
In this paper, we propose a Topical PageRank based algorithm for recommender systems, which aim to rank products by analyzing previous user-item relationships, and recommend top-rank items to potentially interested users. We evaluate our algorithm on MovieLens dataset and empirical experiments demonstrate that it outperforms other state-of-the-art recommending algorithms.
11313782
Efficient Heuristics for Solving Probabilistic Interval Algebra Networks
A probabilistic interval algebra (PIA) network is an interval algebra network with probabilities associated with the labels on an edge. The probabilities on each edge sum to 1. A solution is a consistent scenario where the product of the probabilities associated with each unique edge label is maximized. In this paper we investigate previous PIA network solution algorithms, and propose new ones. Our first algorithm is based on best first search and guarantees to output the optimal solution. However, this algorithm is only feasible for toy problems. We augment the algorithm with three heuristics. Although our proposed algorithm does not guarantee an optimal solution, it is very useful in practice. Good solutions can be generated quickly
2754791
Interference Suppression for Block Diagonalization MIMO Downlink Systems over Time-Varying Channels
The performance of multiuser MIMO downlink systems with block diagonalization (BD) relies on the channel state information (CSI) at the transmitter to a great extent. For time division duplex TDD systems, the transmitter estimates the CSI while receiving data at current time slot and then uses the CSI to transmit at the next time slot. When the wireless channel is time-varying, the CSI for transmission is imperfect due to the time delay between the estimation of the channel and the transmission of the data and severely degrades the system performance. In this paper, we propose a linear method to suppress the interferences among users and data streams caused by imperfect CSI at transmitter. The transmitter first sends pilot signals through a linear spatial precoding matrix so as to make possible that the receiver can estimate CSI of other users, and then the receiver exploits a linear prefilter to suppress the interference. The numerical results show that the proposed schemes achieve obvious performance enhancement in comparison to the BD scheme with imperfect CSI at the transmitter.
32973192
Study on the Biconical Antenna in the EMC Test Suitable for Engineering Machinery