A survey on heterogeneous transfer learning <s> aPLSA <s> The main purpose of this paper is to describe a process for partitioning an N-dimensional population into k sets on the basis of a sample. The process, which is called 'k-means,' appears to give partitions which are reasonably efficient in the sense of within-class variance. That is, if p is the probability mass function for the population, S = {S1, S2, * *, Sk} is a partition of EN, and ui, i = 1, 2, * , k, is the conditional mean of p over the set Si, then W2(S) = ff=ISi f z u42 dp(z) tends to be low for the partitions S generated by the method. We say 'tends to be low,' primarily because of intuitive considerations, corroborated to some extent by mathematical analysis and practical computational experience. Also, the k-means procedure is easily programmed and is computationally economical, so that it is feasible to process very large samples on a digital computer. Possible applications include methods for similarity grouping, nonlinear prediction, approximating multivariate distributions, and nonparametric tests for independence among several variables. In addition to suggesting practical classification methods, the study of k-means has proved to be theoretically interesting. The k-means concept represents a generalization of the ordinary sample mean, and one is naturally led to study the pertinent asymptotic behavior, the object being to establish some sort of law of large numbers for the k-means. This problem is sufficiently interesting, in fact, for us to devote a good portion of this paper to it. The k-means are defined in section 2.1, and the main results which have been obtained on the asymptotic behavior are given there. The rest of section 2 is devoted to the proofs of these results. Section 3 describes several specific possible applications, and reports some preliminary results from computer experiments conducted to explore the possibilities inherent in the k-means idea. The extension to general metric spaces is indicated briefly in section 4. The original point of departure for the work described here was a series of problems in optimal classification (MacQueen [9]) which represented special <s> BIB001 </s> A survey on heterogeneous transfer learning <s> aPLSA <s> This paper presents a novel statistical method for factor analysis of binary and count data which is closely related to a technique known as Latent Semantic Analysis. In contrast to the latter method which stems from linear algebra and performs a Singular Value Decomposition of co-occurrence tables, the proposed technique uses a generative latent class model to perform a probabilistic mixture decomposition. This results in a more principled approach with a solid foundation in statistical inference. More precisely, we propose to make use of a temperature controlled version of the Expectation Maximization algorithm for model fitting, which has shown excellent performance in practice. Probabilistic Latent Semantic Analysis has many applications, most prominently in information retrieval, natural language processing, machine learning from text, and in related areas. The paper presents perplexity results for different types of text and linguistic data collections and discusses an application in automated document indexing. The experiments indicate substantial and consistent improvements of the probabilistic method over standard Latent Semantic Analysis. <s> BIB002 </s> A survey on heterogeneous transfer learning <s> aPLSA <s> This paper focuses on a new clustering task, called self-taught clustering. Self-taught clustering is an instance of unsupervised transfer learning, which aims at clustering a small collection of target unlabeled data with the help of a large amount of auxiliary unlabeled data. The target and auxiliary data can be different in topic distribution. We show that even when the target data are not sufficient to allow effective learning of a high quality feature representation, it is possible to learn the useful features with the help of the auxiliary data on which the target data can be clustered effectively. We propose a co-clustering based self-taught clustering algorithm to tackle this problem, by clustering the target and auxiliary data simultaneously to allow the feature representation from the auxiliary data to influence the target data through a common set of features. Under the new data representation, clustering on the target data can be improved. Our experiments on image clustering show that our algorithm can greatly outperform several state-of-the-art clustering methods when utilizing irrelevant unlabeled auxiliary data. <s> BIB003 </s> A survey on heterogeneous transfer learning <s> aPLSA <s> In this paper, we present a new learning scenario, heterogeneous transfer learning, which improves learning performance when the data can be in different feature spaces and where no correspondence between data instances in these spaces is provided. In the past, we have classified Chinese text documents using English training data under the heterogeneous transfer learning framework. In this paper, we present image clustering as an example to illustrate how unsupervised learning can be improved by transferring knowledge from auxiliary heterogeneous data obtained from the social Web. Image clustering is useful for image sense disambiguation in query-based image search, but its quality is often low due to imagedata sparsity problem. We extend PLSA to help transfer the knowledge from social Web data, which have mixed feature representations. Experiments on image-object clustering and scene clustering tasks show that our approach in heterogeneous transfer learning based on the auxiliary data is indeed effective and promising. <s> BIB004 </s> A survey on heterogeneous transfer learning <s> aPLSA <s> Probabilistic Latent Semantic Analysis is a novel statistical technique for the analysis of two-mode and co-occurrence data, which has applications in information retrieval and filtering, natural language processing, machine learning from text, and in related areas. Compared to standard Latent Semantic Analysis which stems from linear algebra and performs a Singular Value Decomposition of co-occurrence tables, the proposed method is based on a mixture decomposition derived from a latent class model. This results in a more principled approach which has a solid foundation in statistics. In order to avoid overfitting, we propose a widely applicable generalization of maximum likelihood model fitting by tempered EM. Our approach yields substantial and consistent improvements over Latent Semantic Analysis in a number of experiments. <s> BIB005
Yang et al. BIB004 proposed a method for unsupervised HTL for image clustering tasks called annotation-based PLSA (aPLSA). aPLSA performs these tasks through the use of user-annotated auxiliary images from social websites. This method does not require a labeled source or target, but rather utilizes such annotated images to find a correlation between the text and image features. This is to improve the performance of image clustering tasks even when the data is sparse as to learn a latent feature representation. Image clustering tasks aim to organize images into groups, or clusters, based on their similarities or differences. Similar images are grouped into the same cluster, while dissimilar ones are separated into other clusters. A major application for this task is when organizing search-engine results for a query-based image search. Motivated by such application, this method proposes to use heterogeneous transfer learning as to transfer knowledge from socially annotated auxiliary source images to enhance a target clustering task. Correspondences between the instances of the domains are not assumed in this case when using the heterogeneous auxiliary data. Therefore, the images may be unrelated and still improve performance. The proposed method extends the use of Probabilistic Latent Semantic Analysis (PLSA) BIB005 BIB002 to incorporate annotated auxiliary information. The idea is to perform PLSA on both the source and target simultaneously and link them through common latent variables. First, PLSA is applied to the target images as to obtain an image instance-to-feature co-occurrence matrix. Along with this, PLSA is conducted on the annotated image data at the same time to obtain a text-to-image feature co-occurrence matrix. These matrices are used to estimate the clustering function. While performing these simultaneously, the common latent variables are used to combine the two to learn high quality latent variables for use during target clustering. In other words, the image features are clustered into latent variables and then, simultaneously, the annotated auxiliary information is clustered in to the same latent variables. Thus, providing a connection for the transferring of knowledge with an enhanced feature representation for a target image clustering task. Experiments were performed for image clustering comparing the proposed aPLSA to K-means BIB001 , PLSA BIB005 , and STC BIB003 . For K-means and PLSA, experiments were performed applying these methods to the target data only, as well as combining the target and auxiliary data. The results showed that the proposed method had better results than the baselines as it had lower entropy, indicating less randomness for better classification results, under all experiments. This shows that one can use aPLSA to improve an image clustering task by transferring knowledge from unrelated auxiliary annotated images.