A survey on heterogeneous transfer learning <s> Paper overview/contributions <s> A major assumption in many machine learning and data mining algorithms is that the training and future data must be in the same feature space and have the same distribution. However, in many real-world applications, this assumption may not hold. For example, we sometimes have a classification task in one domain of interest, but we only have sufficient training data in another domain of interest, where the latter data may be in a different feature space or follow a different data distribution. In such cases, knowledge transfer, if done successfully, would greatly improve the performance of learning by avoiding much expensive data-labeling efforts. In recent years, transfer learning has emerged as a new learning framework to address this problem. This survey focuses on categorizing and reviewing the current progress on transfer learning for classification, regression, and clustering problems. In this survey, we discuss the relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift. We also explore some potential future issues in transfer learning research. <s> BIB001 </s> A survey on heterogeneous transfer learning <s> Paper overview/contributions <s> Transfer learning aims to improve performance on a target task by utilizing previous knowledge learned from source tasks. In this paper we introduce a novel heterogeneous transfer learning technique, Feature-Space Remapping (FSR), which transfers knowledge between domains with different feature spaces. This is accomplished without requiring typical feature-feature, feature instance, or instance-instance co-occurrence data. Instead we relate features in different feature-spaces through the construction of metafeatures. We show how these techniques can utilize multiple source datasets to construct an ensemble learner which further improves performance. We apply FSR to an activity recognition problem and a document classification problem. The ensemble technique is able to outperform all other baselines and even performs better than a classifier trained using a large amount of labeled data in the target domain. These problems are especially difficult because, in addition to having different feature-spaces, the marginal probability distributions and the class labels are also different. This work extends the state of the art in transfer learning by considering large transfer across dramatically different spaces. <s> BIB002 </s> A survey on heterogeneous transfer learning <s> Paper overview/contributions <s> Machine learning and data mining techniques have been used in numerous real-world applications. An assumption of traditional machine learning methodologies is the training data and testing data are taken from the same domain, such that the input feature space and data distribution characteristics are the same. However, in some real-world machine learning scenarios, this assumption does not hold. There are cases where training data is expensive or difficult to collect. Therefore, there is a need to create high-performance learners trained with more easily obtained data from different domains. This methodology is referred to as transfer learning. This survey paper formally defines transfer learning, presents information on current solutions, and reviews applications applied to transfer learning. Lastly, there is information listed on software downloads for various transfer learning solutions and a discussion of possible future research work. The transfer learning solutions surveyed are independent of data size and can be applied to big data environments. <s> BIB003 </s> A survey on heterogeneous transfer learning <s> Paper overview/contributions <s> Transfer learning addresses the problem of how to leverage previously acquired knowledge (a source domain) to improve the efficiency of learning in a new domain (the target domain). Although transfer learning has been widely researched in the last decade, existing research still has two restrictions: 1) the feature spaces of the domains must be homogeneous; and 2) the target domain must have at least a few labeled instances. These restrictions significantly limit transfer learning models when transferring knowledge across domains, especially in the big data era. To completely break through both of these bottlenecks, a theorem for reliable unsupervised knowledge transfer is proposed to avoid negative transfers, and a Grassmann manifold is applied to measure the distance between heterogeneous feature spaces. Based on this theorem and the Grassmann manifold, this study proposes two heterogeneous unsupervised knowledge transfer (HeUKT) models - known as RLG and GLG. The RLG uses a linear monotonic map (LMM) to reliably project two heterogeneous feature spaces onto a latent feature space and applies geodesic flow kernel (GFK) model to transfers knowledge between two the projected domains. The GLG optimizes the LMM to achieve the highest possible accuracy and guarantees that the geometric properties of the domains remain unchanged during the transfer process. To test the overall effectiveness of two models, this paper reorganizes five public datasets into ten heterogeneous cross-domain tasks across three application fields: credit assessment, text classification, and cancer detection. Extensive experiments demonstrate that the proposed models deliver superior performance over current benchmarks, and that these HeUKT models are a promising way to give computers the associative ability to judge unknown things using related known knowledge. <s> BIB004
In this section, we will review the notations and organization used in this paper. The motivation for this survey paper is to provide a comprehensive, centralized overview of current heterogeneous transfer learning methodologies. The following outlines the contributions of our paper. • Our main contribution is a comprehensive survey of 38 methods for heterogeneous transfer learning which operate under varied settings, requirements, and domains. This provides the centralized outlook into current methodologies. • Second, we present an in-depth discussion and analysis of the surveyed methods. This is done via review and comparison performed amongst the methodologies as well as their limitations. • Third, we present the shortcomings of current research in this domain as well as future work for further investigation. Unlike previous surveys related to transfer learning, we focus distinctly on heterogeneous transfer learning and its related challenges. Specifically, BIB001 only briefly introduces the idea of HTL without surveying any of its methodologies while our survey goes indepth into HTL, its challenges, and its current methods. Also, the survey conducted by BIB003 has a greater focus on general transfer learning methodologies and reviews only a few of the current methodologies for HTL, while we survey and analyze over three times as many. Thus, our unique focus on heterogeneous transfer learning provides a more comprehensive study across state-of-the-art methodologies and provides greater insight over other related surveys into this growing domain. When faced with a cross-domain transfer learning task it is important to understand one's requirements as to select an appropriate method. Because labeling efforts are often expensive, one may have little or no available training labels in the target domain. Thus, this paper intuitively organizes the reviewed methods around target and source label requirements so a machine learning practitioner or researcher can easily select an appropriate HTL method based on the availability of labels in their target task. In the literature, it can be noted that there are opposing definitions for terms used to describe label requirements BIB003 . For example, Liu et al. BIB004 use the notation of supervised transfer learning to denote a fully-labeled source domain and a limited labeled target, while unsupervised transfer learning denotes a mostly labeled source with no target domain labels. Conversely, the work of Cook and Feuz BIB002 who use supervised and unsupervised transfer learning to denote the presence or absence of labels respectively only in the source domain. Furthermore, they use the notation of informed and uninformed to denote the presence or absence of labels respectively in the target domain. Due to these inconsistencies, we denote the categories explicitly by their label requirements to avoid confusion. The surveyed HTL methods are organized into the following categories: "Methods which require limited target labels", "Methods which require limited target labels and accept unlabeled target instances", "Methods which require no target labels", "Methods which require limited target labels and no source labels", "Methods which no target or source labels", as well as an "HTL preprocessing method". In Tables 1,  2 , 3, 4, we provide an overview of the surveyed methods in these categories. Furthermore, we present a deeper analysis into these methods in the "Discussion" section, followed by the "Conclusion", and finalized with "Future work".