A survey on heterogeneous transfer learning <s> CT-SVM <s> Concepts of correlation and regression may be applied not only to ordinary one-dimensional variates but also to variates of two or more dimensions. Marksmen side by side firing simultaneous shots at targets, so that the deviations are in part due to independent individual errors and in part to common causes such as wind, provide a familiar introduction to the theory of correlation; but only the correlation of the horizontal components is ordinarily discussed, whereas the complex consisting of horizontal and vertical deviations may be even more interesting. The wind at two places may be compared, using both components of the velocity in each place. A fluctuating vector is thus matched at each moment with another fluctuating vector. The study of individual differences in mental and physical traits calls for a detailed study of the relations between sets of correlated variates. For example the scores on a number of mental tests may be compared with physical measurements on the same persons. The questions then arise of determining the number and nature of the independent relations of mind and body shown by these data to exist, and of extracting from the multiplicity of correlations in the system suitable characterizations of these independent relations. As another example, the inheritance of intelligence in rats might be studied by applying not one but s different mental tests to N mothers and to a daughter of each <s> BIB001 </s> A survey on heterogeneous transfer learning <s> CT-SVM <s> We present a novel domain adaptation approach for solving cross-domain pattern recognition problems, i.e., the data or features to be processed and recognized are collected from different domains of interest. Inspired by canonical correlation analysis (CCA), we utilize the derived correlation subspace as a joint representation for associating data across different domains, and we advance reduced kernel techniques for kernel CCA (KCCA) if nonlinear correlation subspace are desirable. Such techniques not only makes KCCA computationally more efficient, potential over-fitting problems can be alleviated as well. Instead of directly performing recognition in the derived CCA subspace (as prior CCA-based domain adaptation methods did), we advocate the exploitation of domain transfer ability in this subspace, in which each dimension has a unique capability in associating cross-domain data. In particular, we propose a novel support vector machine (SVM) with a correlation regularizer, named correlation-transfer SVM, which incorporates the domain adaptation ability into classifier design for cross-domain recognition. We show that our proposed domain adaptation and classification approach can be successfully applied to a variety of cross-domain recognition tasks such as cross-view action recognition, handwritten digit recognition with different features, and image-to-text or text-to-image classification. From our empirical results, we verify that our proposed method outperforms state-of-the-art domain adaptation approaches in terms of recognition performance. <s> BIB002
Yeh et al. BIB002 proposed a framework for cross-domain pattern recognition tasks which uses Kernel CCA for symmetric feature transformation and proposed a modified SVM named Correlation-Transfer SVM (we denote as CT-SVM). This method utilizes Canonical Correlation Analysis (CCA) BIB001 for deriving a domain-invariant joint feature space to associate data from the source and target. Here, CCA aims to maximize the correlation, through a coefficient ρ, between two variable sets and learns projection vectors to derive a subspace which can be used as a common feature representation to solve cross-domain problems. Reduced Kernel CCA is utilized when non-linear subspaces are desired. The common subspace dimension is bounded by the minimum feature dimension whether it from the source or target feature space as to prevent overfitting. Once this common subspace is derived, one can project unseen test data onto the subspace for classification by a model trained from the source view data, as in this case no labeled target instances are available. Rather than directly using this subspace, this method proposes to exploit the domain adaption ability that can be observed in the subspace. Each dimension in the derived CCA is associated to its own ρ value. The higher the correlation coefficient ρ, the closer the source and target domains are to each other in such dimension thus it is more suited for domain transfer. An example of this can be seen in Fig. 4 which was inspired from BIB002 . As illustrated in Fig. 4a , in the dimension with the higher ρ value the source and target data are more similar and their points on the subspace are more clustered together. Therefore, a classifier projected onto the subspace will have an easier time discriminating between the classes for the unseen target test data. On the other hand, in Fig. 4b , in the dimension with the lower ρ value the source and target are dissimilar, as there is a higher difference in distribution, which will result in lower recognition performance. Standard CCA-based approaches for Fig. 4 Illustration for CT-SVM, inspired from BIB002 , showing the difference between a ρ value with higher correlation (a) versus a ρ value with lower correlation (b) when projecting source and target instances onto correlation subspaces cross-domain tasks do not take the domain transferability into account when learning classifiers in the correlation subspace which can lead to degraded performance when lower ρ values are present. This method thus proposes using a modified SVM called Correlation-Transfer SVM which takes such transferability into account by enforcing a suppression of the learned SVM along such dimensions with lower ρ values as they have little or no beneficial knowledge for transfer to the target task. This, in effect, will provide more focus on the attributes with higher correlation between the source and target domains where beneficial discriminatory knowledge is more likely to be transferred from. This method does not require labeled target data and it utilizes the available unlabeled target and source pairs to select a kernel parameter which results in the highest correlation performance. Experiments were performed for cross-view action recognition, handwritten digit recognition, and image-to-text/text-to-image classification to analyze the effectiveness of the proposed algorithm. The proposed method was compared with combinations of linear and kernel SVM and CCA where the proposed method was shown to be effective. The results also indicate that deriving a common subspace better suited for domain adaption is more important than designing a complex classifier in a standard feature space.