A survey on heterogeneous transfer learning <s> RLG, GLG <s> In real-world applications of visual recognition, many factors — such as pose, illumination, or image quality — can cause a significant mismatch between the source domain on which classifiers are trained and the target domain to which those classifiers are applied. As such, the classifiers often perform poorly on the target domain. Domain adaptation techniques aim to correct the mismatch. Existing approaches have concentrated on learning feature representations that are invariant across domains, and they often do not directly exploit low-dimensional structures that are intrinsic to many vision datasets. In this paper, we propose a new kernel-based method that takes advantage of such structures. Our geodesic flow kernel models domain shift by integrating an infinite number of subspaces that characterize changes in geometric and statistical properties from the source to the target domain. Our approach is computationally advantageous, automatically inferring important algorithmic parameters without requiring extensive cross-validation or labeled data from either domain. We also introduce a metric that reliably measures the adaptability between a pair of source and target domains. For a given target domain and several source domains, the metric can be used to automatically select the optimal source domain to adapt and avoid less desirable ones. Empirical studies on standard datasets demonstrate the advantages of our approach over competing methods. <s> BIB001 </s> A survey on heterogeneous transfer learning <s> RLG, GLG <s> Domain adaptation aims to correct the mismatch in statistical properties between the source domain on which a classifier is trained and the target domain to which the classifier is to be applied. In this paper, we address the challenging scenario of unsupervised domain adaptation, where the target domain does not provide any annotated data to assist in adapting the classifier. Our strategy is to learn robust features which are resilient to the mismatch across domains and then use them to construct classifiers that will perform well on the target domain. To this end, we propose novel kernel learning approaches to infer such features for adaptation. Concretely, we explore two closely related directions. In the first direction, we propose unsupervised learning of a geodesic flow kernel (GFK). The GFK summarizes the inner products in an infinite sequence of feature subspaces that smoothly interpolates between the source and target domains. In the second direction, we propose supervised learning of a kernel that discriminatively combines multiple base GFKs. Those base kernels model the source and the target domains at fine-grained granularities. In particular, each base kernel pivots on a different set of landmarks--the most useful data instances that reveal the similarity between the source and the target domains, thus bridging them to achieve adaptation. Our approaches are computationally convenient, automatically infer important hyper-parameters, and are capable of learning features and classifiers discriminatively without demanding labeled data from the target domain. In extensive empirical studies on standard benchmark recognition datasets, our appraches yield state-of-the-art results compared to a variety of competing methods. <s> BIB002 </s> A survey on heterogeneous transfer learning <s> RLG, GLG <s> We present a novel domain adaptation approach for solving cross-domain pattern recognition problems, i.e., the data or features to be processed and recognized are collected from different domains of interest. Inspired by canonical correlation analysis (CCA), we utilize the derived correlation subspace as a joint representation for associating data across different domains, and we advance reduced kernel techniques for kernel CCA (KCCA) if nonlinear correlation subspace are desirable. Such techniques not only makes KCCA computationally more efficient, potential over-fitting problems can be alleviated as well. Instead of directly performing recognition in the derived CCA subspace (as prior CCA-based domain adaptation methods did), we advocate the exploitation of domain transfer ability in this subspace, in which each dimension has a unique capability in associating cross-domain data. In particular, we propose a novel support vector machine (SVM) with a correlation regularizer, named correlation-transfer SVM, which incorporates the domain adaptation ability into classifier design for cross-domain recognition. We show that our proposed domain adaptation and classification approach can be successfully applied to a variety of cross-domain recognition tasks such as cross-view action recognition, handwritten digit recognition with different features, and image-to-text or text-to-image classification. From our empirical results, we verify that our proposed method outperforms state-of-the-art domain adaptation approaches in terms of recognition performance. <s> BIB003 </s> A survey on heterogeneous transfer learning <s> RLG, GLG <s> Transfer learning addresses the problem of how to leverage previously acquired knowledge (a source domain) to improve the efficiency of learning in a new domain (the target domain). Although transfer learning has been widely researched in the last decade, existing research still has two restrictions: 1) the feature spaces of the domains must be homogeneous; and 2) the target domain must have at least a few labeled instances. These restrictions significantly limit transfer learning models when transferring knowledge across domains, especially in the big data era. To completely break through both of these bottlenecks, a theorem for reliable unsupervised knowledge transfer is proposed to avoid negative transfers, and a Grassmann manifold is applied to measure the distance between heterogeneous feature spaces. Based on this theorem and the Grassmann manifold, this study proposes two heterogeneous unsupervised knowledge transfer (HeUKT) models - known as RLG and GLG. The RLG uses a linear monotonic map (LMM) to reliably project two heterogeneous feature spaces onto a latent feature space and applies geodesic flow kernel (GFK) model to transfers knowledge between two the projected domains. The GLG optimizes the LMM to achieve the highest possible accuracy and guarantees that the geometric properties of the domains remain unchanged during the transfer process. To test the overall effectiveness of two models, this paper reorganizes five public datasets into ten heterogeneous cross-domain tasks across three application fields: credit assessment, text classification, and cancer detection. Extensive experiments demonstrate that the proposed models deliver superior performance over current benchmarks, and that these HeUKT models are a promising way to give computers the associative ability to judge unknown things using related known knowledge. <s> BIB004
Liu et al. BIB004 proposed two models called Random Linear monotonic map Geodesic flow kernel model (RLG) and Grassmann Linear monotonic map Geodesic flow kernel model (GLG). These methods are designed for unsupervised HTL and use a labeled source domain along with an unlabeled target domain. Both models use a symmetric transformation to map the source and target data into a latent common subspace using a proposed Linear Monotonic Map (LMM) method. To ensure reliability and guarantee that knowledge is transferred appropriately, the resulting mappings must satisfy two conditions: (1) the mappings are monotonic and (2) performing the inverse of the maps results in the original feature spaces. Also, when transferring knowledge, it is important to maintain the same label space of the target domain after transformation as a significant change implies an error must exist. Therefore, in this method, the mappings must also satisfy the condition that the label space remains unchanged in this process. By using these rules when learning the mappings, one avoids particular conditions for negative transfer. From this latent subspace, both models employ the Geodesic Flow Kernel (GFK) BIB002 BIB001 . This is a homogeneous transfer learning method which is being used in this case to transfer knowledge between the projected domains. The difference between the two proposed models lies within the generation of the transformation parameters for the LMM method. Random LMM GFK (RLG) randomly selects the parameters for the mappings. While this is computationally effective, it does not ensure a reliable model. Because of this, GLG is proposed which optimizes these parameters. Grassmann LMM GFK (GLG) uses a Grassmann manifold to measure the distance between the heterogeneous source and target domains as to optimize the parameters of the LMM through the construction of a cost function. The incorporation of the Grassmann manifold also ensures the geometric properties of the data remains unchanged during this process. The downside is that this method is computationally complex and has poor scalability. Optimization of such is done with an evolutionary algorithm through micro-analysis. Experiments were performed for credit assessment, text classification, and cancer detection to test the effectiveness of the proposed RLG and GLG. The proposed methods were compared with variations of KCCA BIB003 , and GFK. The results indicated that both of the proposed methods achieve superior accuracy over the baselines for the tests conducted.