A survey on heterogeneous transfer learning <s> FSR (IFSR, UFSR, ELFSR) <s> This paper introduces stacked generalization, a scheme for minimizing the generalization error rate of one or more generalizers. Stacked generalization works by deducing the biases of the generalizer(s) with respect to a provided learning set. This deduction proceeds by generalizing in a second space whose inputs are (for example) the guesses of the original generalizers when taught with part of the learning set and trying to guess the rest of it, and whose output is (for example) the correct guess. When used with multiple generalizers, stacked generalization can be seen as a more sophisticated version of cross-validation, exploiting a strategy more sophisticated than cross-validation's crude winner-takes-all for combining the individual generalizers. When used with a single generalizer, stacked generalization is a scheme for estimating (and then correcting for) the error of a generalizer which has been trained on a particular learning set and then asked a particular question. After introducing stacked generalization and justifying its use, this paper presents two numerical experiments. The first demonstrates how stacked generalization improves upon a set of separate generalizers for the NETtalk task of translating text to phonemes. The second demonstrates how stacked generalization improves the performance of a single surface-fitter. With the other experimental evidence in the literature, the usual arguments supporting cross-validation, and the abstract justifications presented in this paper, the conclusion is that for almost any real-world generalization problem one should use some version of stacked generalization to minimize the generalization error rate. This paper ends by discussing some of the variations of stacked generalization, and how it touches on other fields like chaos theory. <s> BIB001 </s> A survey on heterogeneous transfer learning <s> FSR (IFSR, UFSR, ELFSR) <s> By their very nature, Intelligent Environments (IE’s) are infused with complexity, unreliability and uncertainty due to a combination of sensor noise and the human element. The quantity, type and availability of data to model these applications can be a major issue. Each situation is contextually different and constantly changing. The dynamic nature of the implementations present a challenging problem when attempting to model or learn a model of the environment. Training data to construct the model must be within the same feature space and have the same distribution as the target task data, however this is often highly costly and time consuming. There can even be occurrences were a complete lack of labelled target data occurs. It is within these situations that our study is focussed. In this paper we propose a framework to dynamically model IE’s through the use of data sets from differing feature spaces and domains. The framework is constructed using a novel Fuzzy Transfer Learning (FuzzyTL) process. <s> BIB002 </s> A survey on heterogeneous transfer learning <s> FSR (IFSR, UFSR, ELFSR) <s> Purpose – The purpose of this paper is to study heterogeneous transfer learning for activity recognition using heuristic search techniques. Many pervasive computing applications require information about the activities currently being performed, but activity recognition algorithms typically require substantial amounts of labeled training data for each setting. One solution to this problem is to leverage transfer learning techniques to reuse available labeled data in new situations. Design/methodology/approach – This paper introduces three novel heterogeneous transfer learning techniques that reverse the typical transfer model and map the target feature space to the source feature space and apply them to activity recognition in a smart apartment. This paper evaluates the techniques on data from 18 different smart apartments located in an assisted-care facility and compares the results against several baselines. Findings – The three transfer learning techniques are all able to outperform the baseline compar... <s> BIB003 </s> A survey on heterogeneous transfer learning <s> FSR (IFSR, UFSR, ELFSR) <s> Transfer learning aims to improve performance on a target task by utilizing previous knowledge learned from source tasks. In this paper we introduce a novel heterogeneous transfer learning technique, Feature-Space Remapping (FSR), which transfers knowledge between domains with different feature spaces. This is accomplished without requiring typical feature-feature, feature instance, or instance-instance co-occurrence data. Instead we relate features in different feature-spaces through the construction of metafeatures. We show how these techniques can utilize multiple source datasets to construct an ensemble learner which further improves performance. We apply FSR to an activity recognition problem and a document classification problem. The ensemble technique is able to outperform all other baselines and even performs better than a classifier trained using a large amount of labeled data in the target domain. These problems are especially difficult because, in addition to having different feature-spaces, the marginal probability distributions and the class labels are also different. This work extends the state of the art in transfer learning by considering large transfer across dramatically different spaces. <s> BIB004
Feuz and Cook BIB004 developed three variants of their proposed Feature-Space Remapping (FSR) method for HTL tasks when one has either limited target labels or optionally no target labels as well as an ensemble technique. This work extends from the previous version of their paper BIB003 . These methods do not assume the source and target domains contain the same feature spaces, probability distributions, dimensions, or label spaces. The proposed method, called Feature-Space Remapping, handles these issues by remapping the target data from their original feature spaces to that of the source feature space through an asymmetric transformation. This mapping is found through the use of metafeatures which are used to find the similarity between a feature pair. More specifically, first metafeatures are created, based on the availability of target labels, from the source and target data. Then, from these metafeatures, a similarity matrix is constructed by computing a similarity score between the source and target metafeatures. To construct the mapping, feature pairs with the highest similarity scores are selected as to decrease the error when applying target domain data to a previously learned source domain model. This mapping is many to one as many target features can map to one source feature but not the reverse. If multiple target features are mapped to a single source feature, then their values are combined with an aggregation protocol such as BIB002 , which provides an overview of the methodology minimum, maximum, average, etc. This mapping is applied to the target test data which maps its feature space to the most related source features and a final prediction is done with the prediction function trained on the source data. This FSR method can also be used as a preprocessing step to obtain a common feature space for later use with homogeneous transfer learning techniques. Three variants of this method are proposed IFSR, UFSR, and ELFSR. Informed Feature-Space Remapping (IFSR) is used when limited target labels are provided. Thus, domain-independent metafeatures can be constructed with the aid of these labels. The metafeatures are found for this method by computing a feature-label co-occurrence value for each feature from the source and target space by calculating the expected value of a feature based on the given label from the labeled training data. Uniformed FeatureSpace Remapping (UFSR) is used when no labeled target training data is provided. Thus, domain-dependent features are constructed due to the lack of labeled to assist in the mapping process. The metafeatures built for this method are created to model the conditional distributions such that the mapping process will select a feature pair with similar conditional distributions as this implies they are more similar for classifying a label and thus should be mapped together. Ensemble Learning via Feature-Space Remapping (ELFSR) extends these models by accepting multiple source domain data and creates an ensemble learning scenario. Here, an individual mapping from the target domain is learned to each one of the input source domains and a separate base classifier is built for each of these source domains. Combining these base classifiers creates an ensemble model. The final prediction for a class label in this model is calculated by a voting or stacking BIB001 scheme. Experiments were performed for activity recognition and document classification tasks which showed promising results for the proposed methods. These promising results are especially noted for the ensemble classifier as ensemble classification often provides better performance over singular models.