A Survey on Soft Subspace Clustering <s> Transfer Learning <s> A major assumption in many machine learning and data mining algorithms is that the training and future data must be in the same feature space and have the same distribution. However, in many real-world applications, this assumption may not hold. For example, we sometimes have a classification task in one domain of interest, but we only have sufficient training data in another domain of interest, where the latter data may be in a different feature space or follow a different data distribution. In such cases, knowledge transfer, if done successfully, would greatly improve the performance of learning by avoiding much expensive data-labeling efforts. In recent years, transfer learning has emerged as a new learning framework to address this problem. This survey focuses on categorizing and reviewing the current progress on transfer learning for classification, regression, and clustering problems. In this survey, we discuss the relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift. We also explore some potential future issues in transfer learning research. <s> BIB001 </s> A Survey on Soft Subspace Clustering <s> Transfer Learning <s> Latent variable models are powerful dimensionality reduction approaches in machine learning and pattern recognition. However, this kind of methods only works well under a necessary and strict assumption that the training samples and testing samples are independent and identically distributed. When the samples come from different domains, the distribution of the testing dataset will not be identical with the training dataset. Therefore, the performance of latent variable models will be degraded for the reason that the parameters of the training model do not suit for the testing dataset. This case limits the generalization and application of the traditional latent variable models. To handle this issue, a transfer learning framework for latent variable model is proposed which can utilize the distance (or divergence) of the two datasets to modify the parameters of the obtained latent variable model. So we do not need to rebuild the model and only adjust the parameters according to the divergence, which will adopt different datasets. Experimental results on several real datasets demonstrate the advantages of the proposed framework. <s> BIB002 </s> A Survey on Soft Subspace Clustering <s> Transfer Learning <s> Adapting the classifier trained on a source domain to recognize instances from a new target domain is an important problem that is receiving recent attention. In this paper, we present one of the first studies on unsupervised domain adaptation in the context of object recognition, where we have labeled data only from the source domain (and therefore do not have correspondences between object categories across domains). Motivated by incremental learning, we create intermediate representations of data between the two domains by viewing the generative subspaces (of same dimension) created from these domains as points on the Grassmann manifold, and sampling points along the geodesic between them to obtain subspaces that provide a meaningful description of the underlying domain shift. We then obtain the projections of labeled source domain data onto these subspaces, from which a discriminative classifier is learnt to classify projected data from the target domain. We discuss extensions of our approach for semi-supervised adaptation, and for cases with multiple source and target domains, and report competitive results on standard datasets. <s> BIB003 </s> A Survey on Soft Subspace Clustering <s> Transfer Learning <s> Visual domain adaptation addresses the problem of adapting the sample distribution of the source domain to the target domain, where the recognition task is intended but the data distributions are different. In this paper, we present a low-rank reconstruction method to reduce the domain distribution disparity. Specifically, we transform the visual samples in the source domain into an intermediate representation such that each transformed source sample can be linearly reconstructed by the samples of the target domain. Unlike the existing work, our method captures the intrinsic relatedness of the source samples during the adaptation process while uncovering the noises and outliers in the source domain that cannot be adapted, making it more robust than previous methods. We formulate our problem as a constrained nuclear norm and l 2, 1 norm minimization objective and then adopt the Augmented Lagrange Multiplier (ALM) method for the optimization. Extensive experiments on various visual adaptation tasks show that the proposed method consistently and significantly beats the state-of-the-art domain adaptation methods. <s> BIB004 </s> A Survey on Soft Subspace Clustering <s> Transfer Learning <s> Recent work has demonstrated the effectiveness of domain adaptation methods for computer vision applications. In this work, we propose a new multiple source domain adaptation method called Domain Selection Machine (DSM) for event recognition in consumer videos by leveraging a large number of loosely labeled web images from different sources (e.g., Flickr.com and Photosig.com), in which there are no labeled consumer videos. Specifically, we first train a set of SVM classifiers (referred to as source classifiers) by using the SIFT features of web images from different source domains. We propose a new parametric target decision function to effectively integrate the static SIFT features from web images/video keyframes and the spacetime (ST) features from consumer videos. In order to select the most relevant source domains, we further introduce a new data-dependent regularizer into the objective of Support Vector Regression (SVR) using the ∊-insensitive loss, which enforces the target classifier shares similar decision values on the unlabeled consumer videos with the selected source classifiers. Moreover, we develop an alternating optimization algorithm to iteratively solve the target decision function and a domain selection vector which indicates the most relevant source domains. Extensive experiments on three real-world datasets demonstrate the effectiveness of our proposed method DSM over the state-of-the-art by a performance gain up to 46.41%. <s> BIB005
The effectiveness of traditional clustering algorithms depends on the sufficiency of data and information. Most of the algorithms fail when the data available is insufficient for performing the clustering tasks, which is not uncommon in real-world applications. This presents a major challenge for traditional clustering algorithms. A promising approach to deal with the issue is to capitalize transfer learning techniques to improve the performance of the clustering algorithms. Transfer learning and domain adaptive learning BIB001 BIB002 BIB004 BIB005 BIB003 are very attractive approaches in the fields of machine learning for their applicability in the real-world modeling and learning tasks, such as classification, regression and clustering, particularly in situations where the data of the current scene (also commonly named as the target domain) are insufficient, or are difficult to be used for a certain task, while some information of a related scene (also commonly named as the source domain) is available for the task. The conceptual diagram in Fig. 1 illustrates the improvement in clustering performance by exploiting transfer learning. The inset in the left shows that the data in the target domain, delineating the situation where it is difficult to obtain satisfactory clustering result with the data using traditional no-transfer learning based clustering algorithms. However, if a reference scene, i.e., the source domain, is available and the information is taken into consideration (right of Fig.   1 ), more promising clustering result can be obtained. For example, if it known that there are two related domains and the number of clusters in these two domains is expected to be the same, then the induced knowledge in the source domain, such as the cluster centers, can be employed to guide the clustering process in the target domain. It can be seen from this example that the realization of effective knowledge/information transfer mechanisms from the source domain to the target domain is the key to develop transfer clustering algorithms. This issue requires special attention and should be properly addressed in the development of effective transfer learning strategy for SSC. For example, the knowledge induced in the source domain, such as the cluster centers and/or the subspace of the clustering results, should be used appropriately to boost the clustering performance in the target domain. Despite the potential of transfer learning and the extensive study for classification and regression tasks in recent years, the application of transfer learning for clustering remains very scarce, especially for high dimensional data. Therefore, research towards the development of transfer leaning based SSC algorithms is promising. It is expected to yield effective algorithms for high dimensional data clustering analyses where transfer learning is useful to enhance clustering performance.