A survey on heterogeneous transfer learning <s> DCN <s> The motivation of this paper is the application of MCMC model scoring procedures to data mining problems, involving a large number of competing models and other relevant model choice aspects. <s> BIB001 </s> A survey on heterogeneous transfer learning <s> DCN <s> In this paper, we present a new learning scenario, heterogeneous transfer learning, which improves learning performance when the data can be in different feature spaces and where no correspondence between data instances in these spaces is provided. In the past, we have classified Chinese text documents using English training data under the heterogeneous transfer learning framework. In this paper, we present image clustering as an example to illustrate how unsupervised learning can be improved by transferring knowledge from auxiliary heterogeneous data obtained from the social Web. Image clustering is useful for image sense disambiguation in query-based image search, but its quality is often low due to imagedata sparsity problem. We extend PLSA to help transfer the knowledge from social Web data, which have mixed feature representations. Experiments on image-object clustering and scene clustering tasks show that our approach in heterogeneous transfer learning based on the auxiliary data is indeed effective and promising. <s> BIB002 </s> A survey on heterogeneous transfer learning <s> DCN <s> Transfer learning as a new machine learning paradigm has gained increasing attention lately. In situations where the training data in a target domain are not sufficient to learn predictive models effectively, transfer learning leverages auxiliary source data from other related source domains for learning. While most of the existing works in this area only focused on using the source data with the same structure as the target data, in this paper, we push this boundary further by proposing a heterogeneous transfer learning framework for knowledge transfer between text and images. We observe that for a target-domain classification problem, some annotated images can be found on many social Web sites, which can serve as a bridge to transfer knowledge from the abundant text documents available over the Web. A key question is how to effectively transfer the knowledge in the source data even though the text can be arbitrarily found. Our solution is to enrich the representation of the target images with semantic concepts extracted from the auxiliary source data through a novel matrix factorization method. By using the latent semantic features generated by the auxiliary data, we are able to build a better integrated image classifier. We empirically demonstrate the effectiveness of our algorithm on the Caltech-256 image dataset. <s> BIB003 </s> A survey on heterogeneous transfer learning <s> DCN <s> This paper studies a new machine learning strategy called co-transfer learning. Unlike many previous learning problems, we focus on how to use labeled data of different feature spaces to enhance the classification of different learning spaces simultaneously. For instance, we make use of both labeled images and labeled text data to help learn models for classifying image data and text data together. An important component of co-transfer learning is to build different relations to link different feature spaces, thus knowledge can be co-transferred across different spaces. Our idea is to model the problem as a joint transition probability graph. The transition probabilities can be constructed by using the intra-relationships based on affinity metric among instances and the inter-relationships based on co-occurrence information among instances from different spaces. The proposed algorithm computes ranking of labels to indicate the importance of a set of labels to an instance by propagating the ranking score of labeled instances via the random walk with restart. The main contribution of this paper is to (i) propose a co-transfer learning (CT-Learn) framework that can perform learning simultaneously by co-transferring knowledge across different spaces; (ii) show the theoretical properties of the random walk for such joint transition probability graph so that the proposed learning model can be used effectively; (iii) develop an efficient algorithm to compute ranking scores and generate the possible labels for a given instance. Experimental results on benchmark data (image-text and English-Chinese-French classification data sets) have shown that the proposed algorithm is computationally efficient, and effective in learning across different spaces. In the comparison, we find that the classification performance of the CT-Learn algorithm is better than those of the other tested transfer learning algorithms. <s> BIB004 </s> A survey on heterogeneous transfer learning <s> DCN <s> One of the main research problems in heterogeneous transfer learning is to determine whether a given source domain is effective in transferring knowledge to a target domain, and then to determine how much of the knowledge should be transferred from a source domain to a target domain. The main objective of this paper is to solve this problem by evaluating the relatedness among given domains through transferred weights. We propose a novel method to learn such transferred weights with the aid of co-occurrence data, which contain the same set of instances but in different feature spaces. Because instances with the same category should have similar features, our method is to compute their principal components in each feature space such that co-occurrence data can be rerepresented by these principal components. The principal component coefficients from different feature spaces for the same instance in the co-occurrence data have the same order of significance for describing the category information. By using these principal component coefficients, the Markov Chain Monte Carlo method is employed to construct a directed cyclic network where each node is a domain and each edge weight is the conditional dependence from one domain to another domain. Here, the edge weight of the network can be employed as the transferred weight from a source domain to a target domain. The weight values can be taken as a prior for setting parameters in the existing heterogeneous transfer learning methods to control the amount of knowledge transferred from a source domain to a target domain. The experimental results on synthetic and real-world data sets are reported to illustrate the effectiveness of the proposed method that can capture strong or weak relations among feature spaces, and enhance the learning performance of heterogeneous transfer learning. <s> BIB005
Yang et al. BIB005 proposed an HTL preprocessing method based on using a Directed Cyclic Network (DCN). This method is proposed to be used before performing an HTL task to improve the overall effectiveness of the transfer learning method used. This method aims to measure the relatedness between N ≥ 2 heterogeneous domains through co-occurrence data by learning transferred weights in order to determine whether a source domain is suitable for transferring knowledge to a target domain, as well as how much of that knowledge should be transferred. This co-occurrence data represents the same data from the target and source domains, but in different feature spaces. Therefore, one can use it independently to determine similarities amongst them to see whether a domain is suitable for knowledge transfer before applying computationally complex HTL methods. This is done by performing PCA on the co-occurrence data to compute their principal components for each feature space. The coefficients of such components have the same discriminatory order of significance for prediction purposes as the original domain's feature spaces. Using these coefficients, a directed cyclic network is built for each principal component to model the relationship amongst the domains and capture strong or weak relations amongst them. Here, each node represents a domain and the directed edge contains the transferred weight measuring the conditional dependence from one domain to another. The larger the weight value, the more knowledge can be transferred from one domain to another. Note that this graph can be asymmetric, as the i th domain may affect the j th more than the j th affects the i th domain. The optimal network structure is computed through a Markov Chain Monte Carlo (MCMC) BIB001 method. Once a DCN is constructed for each principal component, they are merged into a single DCN by computing a weighted sum of the edge weights. More weight is given to the components used to construct the DCN that have higher contributions which is based on their corresponding eigenvalues. A low final edge weight means little knowledge can be transferred from such source domains implying it is unrelated and not suitable for knowledge transfer thus aiding in selecting the best source domain to use. The final transferred weights can also be used to set the parameters in the HTL methods for better overall performance. This was proven in the experiments under which the proposed method was incorporated into aPLSA BIB002 , HTLIC BIB003 , and CT-Learn BIB004 . Note that these each fall into different label requirement categories, thus implying the proposed method may be used for any HTL scenario.