Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> We consider the scenario where training and test data are drawn from different distributions, commonly referred to as sample selection bias. Most algorithms for this setting try to first recover sampling distributions and then make appropriate corrections based on the distribution estimate. We present a nonparametric method which directly produces resampling weights without distribution estimation. Our method works by matching distributions between training and testing sets in feature space. Experimental results demonstrate that our method works well in practice. <s> BIB001 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Discriminative learning methods are widely used in natural language processing. These methods work best when their training and test data are drawn from the same distribution. For many NLP tasks, however, we are confronted with new domains in which labeled data is scarce or non-existent. In such cases, we seek to adapt existing models from a resource-rich source domain to a resource-poor target domain. We introduce structural correspondence learning to automatically induce correspondences among features from different domains. We test our technique on part of speech tagging and show performance gains for varying amounts of source and target training data, as well as improvements in target domain parsing accuracy using our improved tagger. <s> BIB002 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> A basic assumption in traditional machine learning is that the training and test data distributions should be identical. This assumption may not hold in many situations in practice, but we may be forced to rely on a different-distribution data to learn a prediction model. For example, this may be the case when it is expensive to label the data in a domain of interest, although in a related but different domain there may be plenty of labeled data available. In this paper, we propose a novel transfer-learning algorithm for text classification based on an EM-based Naive Bayes classifiers. Our solution is to first estimate the initial probabilities under a distribution Dl of one labeled data set, and then use an EM algorithm to revise the model for a different distribution Du of the test data which are unlabeled. We show that our algorithm is very effective in several different pairs of domains, where the distances between the different distributions are measured using the Kullback-Leibler (KL) divergence. Moreover, KL-divergence is used to decide the trade-off parameters in our algorithm. In the experiment, our algorithm outperforms the traditional supervised and semi-supervised learning algorithms when the distributions of the training and test sets are increasingly different. <s> BIB003 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> A situation where training and test samples follow different input distributions is called covariate shift. Under covariate shift, standard learning methods such as maximum likelihood estimation are no longer consistent—weighted variants according to the ratio of test and training input densities are consistent. Therefore, accurately estimating the density ratio, called the importance, is one of the key issues in covariate shift adaptation. A naive approach to this task is to first estimate training and test input densities separately and then estimate the importance by taking the ratio of the estimated densities. However, this naive approach tends to perform poorly since density estimation is a hard task particularly in high dimensional cases. In this paper, we propose a direct importance estimation method that does not involve density estimation. Our method is equipped with a natural cross validation procedure and hence tuning parameters such as the kernel width can be objectively optimized. Simulations illustrate the usefulness of our approach. <s> BIB004 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> One common predictive modeling challenge occurs in text mining problems is that the training data and the operational (testing) data are drawn from different underlying distributions. This poses a great difficulty for many statistical learning methods. However, when the distribution in the source domain and the target domain are not identical but related, there may exist a shared concept space to preserve the relation. Consequently a good feature representation can encode this concept space and minimize the distribution gap. To formalize this intuition, we propose a domain adaptation method that parameterizes this concept space by linear transformation under which we explicitly minimize the distribution difference between the source domain with sufficient labeled data and target domains with only unlabeled data, while at the same time minimizing the empirical loss on the labeled data in the source domain. Another characteristic of our method is its capability for considering multiple classes and their interactions simultaneously. We have conducted extensive experiments on two common text mining problems, namely, information extraction and document classification to demonstrate the effectiveness of our proposed method. <s> BIB005 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Recently there has been increasing interest in the problem of transfer learning, in which the typical assumption that training and testing data are drawn from identical distributions is relaxed. We specifically address the problem of transductive transfer learning in which we have access to labeled training data and unlabeled testing data potentially drawn from different, yet related distributions, and the goal is to leverage the labeled training data to learn a classifier to correctly predict data from the testing distribution. We have derived efficient algorithms for transductive transfer learning based on a novel viewpoint and the Support Vector Machine (SVM) paradigm, of a large margin hyperplane classifier in a feature space. We show that our method can out-perform some recent state-of-the-art approaches for transfer learning on several data sets, with the added benefits of model and data separation and the potential to leverage existing work on support vector machines. <s> BIB006 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> In activity recognition, one major challenge is huge manual effort in labeling when a new domain of activities is to be tested. In this paper, we ask an interesting question: can we transfer the available labeled data from a set of existing activities in one domain to help recognize the activities in another different but related domain? Our answer is "yes", provided that the sensor data from the two domains are related in some way. We develop a bridge between the activities in two domains by learning a similarity function via Web search, under the condition that the sensor data are from the same feature space. Based on the learned similarity measures, our algorithm interprets the data from the source domain as the data in the domain with different confidence levels, thus accomplishing the cross-domain knowledge transfer task. Our algorithm is evaluated on several real-world datasets to demonstrate its effectiveness. <s> BIB007 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> We consider the problem of zero-shot learning, where the goal is to learn a classifier f : X → Y that must predict novel values of Y that were omitted from the training set. To achieve this, we define the notion of a semantic output code classifier (SOC) which utilizes a knowledge base of semantic properties of Y to extrapolate to novel classes. We provide a formalism for this type of classifier and study its theoretical properties in a PAC framework, showing conditions under which the classifier can accurately predict novel classes. As a case study, we build a SOC classifier for a neural decoding task and show that it can often predict words that people are thinking about from functional magnetic resonance images (fMRI) of their neural activity, even without training examples for those words. <s> BIB008 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> The regularization principals [31] lead approximation schemes to deal with various learning problems, e.g., the regularization of the norm in a reproducing kernel Hilbert space for the ill-posed problem. In this paper, we present a family of subspace learning algorithms based on a new form of regularization, which transfers the knowledge gained in training samples to testing samples. In particular, the new regularization minimizes the Bregman divergence between the distribution of training samples and that of testing samples in the selected subspace, so it boosts the performance when training and testing samples are not independent and identically distributed. To test the effectiveness of the proposed regularization, we introduce it to popular subspace learning algorithms, e.g., principal components analysis (PCA) for cross-domain face modeling; and Fisher's linear discriminant analysis (FLDA), locality preserving projections (LPP), marginal Fisher's analysis (MFA), and discriminative locality alignment (DLA) for cross-domain face recognition and text categorization. Finally, we present experimental evidence on both face image data sets and text data sets, suggesting that the proposed Bregman divergence-based regularization is effective to deal with cross-domain learning problems. <s> BIB009 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Discriminative learning when training and test data belong to different distributions is a challenging and complex task. Often times we have very few or no labeled data from the test or target distribution but may have plenty of labeled data from multiple related sources with different distributions. The difference in distributions may be both in marginal and conditional probabilities. Most of the existing domain adaptation work focuses on the marginal probability distribution difference between the domains, assuming that the conditional probabilities are similar. However in many real world applications, conditional probability distribution differences are as commonplace as marginal probability differences. In this paper we propose a two-stage domain adaptation methodology which combines weighted data from multiple sources based on marginal probability differences (first stage) as well as conditional probability differences (second stage), with the target domain data. The weights for minimizing the marginal probability differences are estimated independently, while the weights for minimizing conditional probability differences are computed simultaneously by exploiting the potential interaction among multiple sources. We also provide a theoretical analysis on the generalization performance of the proposed multi-source domain adaptation formulation using the weighted Rademacher complexity measure. Empirical comparisons with existing state-of-the-art domain adaptation methods using three real-world datasets demonstrate the effectiveness of the proposed approach. <s> BIB010 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> The state-of-the-art metric-learning algorithms cannot perform well for domain adaptation settings, such as cross-domain face recognition, image annotation, etc., because labeled data in the source domain and unlabeled ones in the target domain are drawn from different, but related distributions. In this paper, we propose the domain adaptation metric learning (DAML), by introducing a data-dependent regularization to the conventional metric learning in the reproducing kernel Hilbert space (RKHS). This data-dependent regularization resolves the distribution difference by minimizing the empirical maximum mean discrepancy between source and target domain data in RKHS. Theoretically, by using the empirical Rademacher complexity, we prove risk bounds for the nearest neighbor classifier that uses the metric learned by DAML. Practically, learning the metric in RKHS does not scale up well. Fortunately, we can prove that learning DAML in RKHS is equivalent to learning DAML in the space spanned by principal components of the kernel principle component analysis (KPCA). Thus, we can apply KPCA to select most important principal components to significantly reduce the time cost of DAML. We perform extensive experiments over four well-known face recognition datasets and a large-scale Web image annotation dataset for the cross-domain face recognition and image annotation tasks under various settings, and the results demonstrate the effectiveness of DAML. <s> BIB011 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Adapting the classifier trained on a source domain to recognize instances from a new target domain is an important problem that is receiving recent attention. In this paper, we present one of the first studies on unsupervised domain adaptation in the context of object recognition, where we have labeled data only from the source domain (and therefore do not have correspondences between object categories across domains). Motivated by incremental learning, we create intermediate representations of data between the two domains by viewing the generative subspaces (of same dimension) created from these domains as points on the Grassmann manifold, and sampling points along the geodesic between them to obtain subspaces that provide a meaningful description of the underlying domain shift. We then obtain the projections of labeled source domain data onto these subspaces, from which a discriminative classifier is learnt to classify projected data from the target domain. We discuss extensions of our approach for semi-supervised adaptation, and for cases with multiple source and target domains, and report competitive results on standard datasets. <s> BIB012 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> The exponential increase in the availability of online reviews and recommendations makes sentiment classification an interesting topic in academic and industrial research. Reviews can span so many different domains that it is difficult to gather annotated training data for all of them. Hence, this paper studies the problem of domain adaptation for sentiment classifiers, hereby a system is trained on labeled reviews from one source domain but is meant to be deployed on another. We propose a deep learning approach which learns to extract a meaningful representation for each review in an unsupervised fashion. Sentiment classifiers trained with this high-level feature representation clearly outperform state-of-the-art methods on a benchmark composed of reviews of 4 types of Amazon products. Furthermore, this method scales well and allowed us to successfully perform domain adaptation on a larger industrial-strength dataset of 22 domains. <s> BIB013 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Activity recognition aims to identify and predict human activities based on a series of sensor readings. In recent years, machine learning methods have become popular in solving activity recognition problems. A special difficulty for adopting machine learning methods is the workload to annotate a large number of sensor readings as training data. Labeling sensor readings for their corresponding activities is a time-consuming task. In practice, we often have a set of labeled training instances ready for an activity recognition task. If we can transfer such knowledge to a new activity recognition scenario that is different from, but related to, the source domain, it will ease our effort to perform manual labeling of training data for the new scenario. In this paper, we propose a transfer learning framework based on automatically learning a correspondence between different sets of sensors to solve this transfer-learning in activity recognition problem. We validate our framework on two different datasets and compare it against previous approaches of activity recognition, and demonstrate its effectiveness. <s> BIB014 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Visual domain adaptation addresses the problem of adapting the sample distribution of the source domain to the target domain, where the recognition task is intended but the data distributions are different. In this paper, we present a low-rank reconstruction method to reduce the domain distribution disparity. Specifically, we transform the visual samples in the source domain into an intermediate representation such that each transformed source sample can be linearly reconstructed by the samples of the target domain. Unlike the existing work, our method captures the intrinsic relatedness of the source samples during the adaptation process while uncovering the noises and outliers in the source domain that cannot be adapted, making it more robust than previous methods. We formulate our problem as a constrained nuclear norm and l 2, 1 norm minimization objective and then adopt the Augmented Lagrange Multiplier (ALM) method for the optimization. Extensive experiments on various visual adaptation tasks show that the proposed method consistently and significantly beats the state-of-the-art domain adaptation methods. <s> BIB015 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> In real-world applications of visual recognition, many factors — such as pose, illumination, or image quality — can cause a significant mismatch between the source domain on which classifiers are trained and the target domain to which those classifiers are applied. As such, the classifiers often perform poorly on the target domain. Domain adaptation techniques aim to correct the mismatch. Existing approaches have concentrated on learning feature representations that are invariant across domains, and they often do not directly exploit low-dimensional structures that are intrinsic to many vision datasets. In this paper, we propose a new kernel-based method that takes advantage of such structures. Our geodesic flow kernel models domain shift by integrating an infinite number of subspaces that characterize changes in geometric and statistical properties from the source to the target domain. Our approach is computationally advantageous, automatically inferring important algorithmic parameters without requiring extensive cross-validation or labeled data from either domain. We also introduce a metric that reliably measures the adaptability between a pair of source and target domains. For a given target domain and several source domains, the metric can be used to automatically select the optimal source domain to adapt and avoid less desirable ones. Empirical studies on standard datasets demonstrate the advantages of our approach over competing methods. <s> BIB016 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> In visual recognition problems, the common data distribution mismatches between training and testing make domain adaptation essential. However, image data is difficult to manually divide into the discrete domains required by adaptation algorithms, and the standard practice of equating datasets with domains is a weak proxy for all the real conditions that alter the statistics in complex ways (lighting, pose, background, resolution, etc.) We propose an approach to automatically discover latent domains in image or video datasets. Our formulation imposes two key properties on domains: maximum distinctiveness and maximum learnability. By maximum distinctiveness, we require the underlying distributions of the identified domains to be different from each other to the maximum extent; by maximum learnability, we ensure that a strong discriminative model can be learned from the domain. We devise a nonparametric formulation and efficient optimization procedure that can successfully discover domains among both training and test data. We extensively evaluate our approach on object recognition and human activity recognition tasks. <s> BIB017 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Domain-invariant representations are key to addressing the domain shift problem where the training and test examples follow different distributions. Existing techniques that have attempted to match the distributions of the source and target domains typically compare these distributions in the original feature space. This space, however, may not be directly suitable for such a comparison, since some of the features may have been distorted by the domain shift, or may be domain specific. In this paper, we introduce a Domain Invariant Projection approach: An unsupervised domain adaptation method that overcomes this issue by extracting the information that is invariant across the source and target domains. More specifically, we learn a projection of the data to a low-dimensional latent space where the distance between the empirical distributions of the source and target examples is minimized. We demonstrate the effectiveness of our approach on the task of visual object recognition and show that it outperforms state-of-the-art methods on a standard domain adaptation benchmark dataset. <s> BIB018 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> In this paper, we propose a novel method for cross-view action recognition via a continuous virtual path which connects the source view and the target view. Each point on this virtual path is a virtual view which is obtained by a linear transformation of the action descriptor. All the virtual views are concatenated into an infinite-dimensional feature to characterize continuous changes from the source to the target view. However, these infinite-dimensional features cannot be used directly. Thus, we propose a virtual view kernel to compute the value of similarity between two infinite-dimensional features, which can be readily used to construct any kernelized classifiers. In addition, there are a lot of unlabeled samples from the target view, which can be utilized to improve the performance of classifiers. Thus, we present a constraint strategy to explore the information contained in the unlabeled samples. The rationality behind the constraint is that any action video belongs to only one class. Our method is verified on the IXMAS dataset, and the experimental results demonstrate that our method achieves better performance than the state-of-the-art methods. <s> BIB019 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Domain adaptation addresses the problem where data instances of a source domain have different distributions from that of a target domain, which occurs frequently in many real life scenarios. This work focuses on unsupervised domain adaptation, where labeled data are only available in the source domain. We propose to interpolate subspaces through dictionary learning to link the source and target domains. These subspaces are able to capture the intrinsic domain shift and form a shared feature representation for cross domain recognition. Further, we introduce a quantitative measure to characterize the shift between two domains, which enables us to select the optimal domain to adapt to the given multiple source domains. We present experiments on face recognition across pose, illumination and blur variations, cross dataset object recognition, and report improved performance over the state of the art. <s> BIB020 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> In this paper, we introduce a new domain adaptation (DA) algorithm where the source and target domains are represented by subspaces described by eigenvectors. In this context, our method seeks a domain adaptation solution by learning a mapping function which aligns the source subspace with the target one. We show that the solution of the corresponding optimization problem can be obtained in a simple closed form, leading to an extremely fast algorithm. We use a theoretical result to tune the unique hyper parameter corresponding to the size of the subspaces. We run our method on various datasets and show that, despite its intrinsic simplicity, it outperforms state of the art DA methods. <s> BIB021 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> We evaluate whether features extracted from the activation of a deep convolutional network trained in a fully supervised fashion on a large, fixed set of object recognition tasks can be repurposed to novel generic tasks. Our generic tasks may differ significantly from the originally trained tasks and there may be insufficient labeled or unlabeled data to conventionally train or adapt a deep architecture to the new tasks. We investigate and visualize the semantic clustering of deep convolutional features with respect to a variety of such tasks, including scene recognition, domain adaptation, and fine-grained recognition challenges. We compare the efficacy of relying on various network levels to define a fixed feature, and report novel results that significantly outperform the state-of-the-art on several important vision challenges. We are releasing DeCAF, an open-source implementation of these deep convolutional activation features, along with all associated network parameters to enable vision researchers to be able to conduct experimentation with deep representations across a range of visual concept learning paradigms. <s> BIB022 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Cross-domain image synthesis and recognition are typically considered as two distinct tasks in the areas of computer vision and pattern recognition. Therefore, it is not clear whether approaches addressing one task can be easily generalized or extended for solving the other. In this paper, we propose a unified model for coupled dictionary and feature space learning. The proposed learning model not only observes a common feature space for associating cross-domain image data for recognition purposes, the derived feature space is able to jointly update the dictionaries in each image domain for improved representation. This is why our method can be applied to both cross-domain image synthesis and recognition problems. Experiments on a variety of synthesis and recognition tasks such as single image super-resolution, cross-view action recognition, and sketch-to-photo face recognition would verify the effectiveness of our proposed learning model. <s> BIB023 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Modern visual recognition systems are often limited in their ability to scale to large numbers of object categories. This limitation is in part due to the increasing difficulty of acquiring sufficient training data in the form of labeled images as the number of object categories grows. One remedy is to leverage data from other sources - such as text data - both to train visual models and to constrain their predictions. In this paper we present a new deep visual-semantic embedding model trained to identify visual objects using both labeled image data as well as semantic information gleaned from unannotated text. We demonstrate that this model matches state-of-the-art performance on the 1000-class ImageNet object recognition challenge while making more semantically reasonable errors, and also show that the semantic information can be exploited to make predictions about tens of thousands of image labels not observed during training. Semantic knowledge improves such zero-shot predictions achieving hit rates of up to 18% across thousands of novel labels never seen by the visual model. <s> BIB024 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> The recently introduced continuous Skip-gram model is an efficient method for learning high-quality distributed vector representations that capture a large number of precise syntactic and semantic word relationships. In this paper we present several extensions that improve both the quality of the vectors and the training speed. By subsampling of the frequent words we obtain significant speedup and also learn more regular word representations. We also describe a simple alternative to the hierarchical softmax called negative sampling. ::: ::: An inherent limitation of word representations is their indifference to word order and their inability to represent idiomatic phrases. For example, the meanings of "Canada" and "Air" cannot be easily combined to obtain "Air Canada". Motivated by this example, we present a simple method for finding phrases in text, and show that learning good vector representations for millions of phrases is possible. <s> BIB025 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> This work introduces a model that can recognize objects in images even if no training data is available for the object class. The only necessary knowledge about unseen visual categories comes from unsupervised text corpora. Unlike previous zero-shot learning models, which can only differentiate between unseen classes, our model can operate on a mixture of seen and unseen classes, simultaneously obtaining state of the art performance on classes with thousands of training images and reasonable performance on unseen classes. This is achieved by seeing the distributions of words in texts as a semantic space for understanding what objects look like. Our deep learning model does not require any manually defined semantic or visual features for either words or images. Images are mapped to be close to semantic word vectors corresponding to their classes, and the resulting image embeddings can be used to distinguish whether an image is of a seen or unseen class. We then use novelty detection methods to differentiate unseen classes from seen classes. We demonstrate two novelty detection strategies; the first gives high accuracy on unseen classes, while the second is conservative in its prediction of novelty and keeps the seen classes' accuracy high. <s> BIB026 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> In this paper, we tackle the problem of unsupervised domain adaptation for classification. In the unsupervised scenario where no labeled samples from the target domain are provided, a popular approach consists in transforming the data such that the source and target distributions become similar. To compare the two distributions, existing approaches make use of the Maximum Mean Discrepancy (MMD). However, this does not exploit the fact that probability distributions lie on a Riemannian manifold. Here, we propose to make better use of the structure of this manifold and rely on the distance on the manifold to compare the source and target distributions. In this framework, we introduce a sample selection method and a subspace-based method for unsupervised domain adaptation, and show that both these manifold-based techniques outperform the corresponding approaches based on the MMD. Furthermore, we show that our subspace-based approach yields state-of-the-art results on a standard object recognition benchmark. <s> BIB027 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> With unconstrained data acquisition scenarios widely prevalent, the ability to handle changes in data distribution across training and testing data sets becomes important. One way to approach this problem is through domain adaptation, and in this paper we primarily focus on the unsupervised scenario where the labeled source domain training data is accompanied by unlabeled target domain test data. We present a two-stage data-driven approach by generating intermediate data representations that could provide relevant information on the domain shift. Starting with a linear representation of domains in the form of generative subspaces of same dimensions for the source and target domains, we first utilize the underlying geometry of the space of these subspaces, the Grassmann manifold, to obtain a `shortest' geodesic path between the two domains. We then sample points along the geodesic to obtain intermediate cross-domain data representations, using which a discriminative classifier is learnt to estimate the labels of the target data. We subsequently incorporate non-linear representation of domains by considering a Reproducing Kernel Hilbert Space representation, and a low-dimensional manifold representation using Laplacian Eigenmaps, and also examine other domain adaptation settings such as (i) semi-supervised adaptation where the target domain is partially labeled, and (ii) multi-domain adaptation where there could be more than one domain in source and/or target data sets. Finally, we supplement our adaptation technique with (i) fine-grained reference domains that are created by blending samples from source and target data sets to provide some evidence on the actual domain shift, and (ii) a multi-class boosting analysis to obtain robustness to the choice of algorithm parameters. We evaluate our approach for object recognition problems and report competitive results on two widely used Office and Bing adaptation data sets. <s> BIB028 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Domain adaptation aims to correct the mismatch in statistical properties between the source domain on which a classifier is trained and the target domain to which the classifier is to be applied. In this paper, we address the challenging scenario of unsupervised domain adaptation, where the target domain does not provide any annotated data to assist in adapting the classifier. Our strategy is to learn robust features which are resilient to the mismatch across domains and then use them to construct classifiers that will perform well on the target domain. To this end, we propose novel kernel learning approaches to infer such features for adaptation. Concretely, we explore two closely related directions. In the first direction, we propose unsupervised learning of a geodesic flow kernel (GFK). The GFK summarizes the inner products in an infinite sequence of feature subspaces that smoothly interpolates between the source and target domains. In the second direction, we propose supervised learning of a kernel that discriminatively combines multiple base GFKs. Those base kernels model the source and the target domains at fine-grained granularities. In particular, each base kernel pivots on a different set of landmarks--the most useful data instances that reveal the similarity between the source and the target domains, thus bridging them to achieve adaptation. Our approaches are computationally convenient, automatically infer important hyper-parameters, and are capable of learning features and classifiers discriminatively without demanding labeled data from the target domain. In extensive empirical studies on standard benchmark recognition datasets, our appraches yield state-of-the-art results compared to a variety of competing methods. <s> BIB029 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Domain adaptation has shown promising results in computer vision applications. In this paper, we propose a new unsupervised domain adaptation method called domain adaptation by shifting covariance (DASC) for object recognition without requiring any labeled samples from the target domain. By characterizing samples from each domain as one covariance matrix, the source and target domain are represented into two distinct points residing on a Riemannian manifold. Along the geodesic constructed from the two points, we then interpolate some intermediate points (i.e., covariance matrices), which are used to bridge the two domains. By utilizing the principal components of each covariance matrix, samples from each domain are further projected into intermediate feature spaces, which finally leads to domain-invariant features after the concatenation of these features from intermediate points. In the multiple source domain adaptation task, we also need to effectively integrate different types of features between each pair of source and target domains. We additionally propose an SVM based method to simultaneously learn the optimal target classifier as well as the optimal weights for different source domains. Extensive experiments demonstrate the effectiveness of our method for both single source and multiple source domain adaptation tasks. <s> BIB030 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> In this paper, we propose a Generalized Unsupervised Manifold Alignment (GU-MA) method to build the connections between different but correlated datasets without any known correspondences. Based on the assumption that datasets of the same theme usually have similar manifold structures, GUMA is formulated into an explicit integer optimization problem considering the structure matching and preserving criteria, as well as the feature comparability of the corresponding points in the mutual embedding space. The main benefits of this model include: (1) simultaneous discovery and alignment of manifold structures; (2) fully unsuper-vised matching without any pre-specified correspondences; (3) efficient iterative alignment without computations in all permutation cases. Experimental results on dataset matching and real-world applications demonstrate the effectiveness and the practicability of our manifold alignment method. <s> BIB031 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a discriminative model D that estimates the probability that a sample came from the training data rather than G. The training procedure for G is to maximize the probability of D making a mistake. This framework corresponds to a minimax two-player game. In the space of arbitrary functions G and D, a unique solution exists, with G recovering the training data distribution and D equal to ½ everywhere. In the case where G and D are defined by multilayer perceptrons, the entire system can be trained with backpropagation. There is no need for any Markov chains or unrolled approximate inference networks during either training or generation of samples. Experiments demonstrate the potential of the framework through qualitative and quantitative evaluation of the generated samples. <s> BIB032 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Many deep neural networks trained on natural images exhibit a curious phenomenon in common: on the first layer they learn features similar to Gabor filters and color blobs. Such first-layer features appear not to be specific to a particular dataset or task, but general in that they are applicable to many datasets and tasks. Features must eventually transition from general to specific by the last layer of the network, but this transition has not been studied extensively. In this paper we experimentally quantify the generality versus specificity of neurons in each layer of a deep convolutional neural network and report a few surprising results. Transferability is negatively affected by two distinct issues: (1) the specialization of higher layer neurons to their original task at the expense of performance on the target task, which was expected, and (2) optimization difficulties related to splitting networks between co-adapted neurons, which was not expected. In an example network trained on ImageNet, we demonstrate that either of these two issues may dominate, depending on whether features are transferred from the bottom, middle, or top of the network. We also document that the transferability of features decreases as the distance between the base task and target task increases, but that transferring features even from distant tasks can be better than using random features. A final surprising result is that initializing a network with transferred features from almost any number of layers can produce a boost to generalization that lingers even after fine-tuning to the target dataset. <s> BIB033 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Top-performing deep architectures are trained on massive amounts of labeled data. In the absence of labeled data for a certain task, domain adaptation often provides an attractive option given that labeled data of similar nature but from a different domain (e.g. synthetic images) are available. Here, we propose a new approach to domain adaptation in deep architectures that can be trained on large amount of labeled data from the source domain and large amount of unlabeled data from the target domain (no labeled target-domain data is necessary). ::: As the training progresses, the approach promotes the emergence of "deep" features that are (i) discriminative for the main learning task on the source domain and (ii) invariant with respect to the shift between the domains. We show that this adaptation behaviour can be achieved in almost any feed-forward model by augmenting it with few standard layers and a simple new gradient reversal layer. The resulting augmented architecture can be trained using standard backpropagation. ::: Overall, the approach can be implemented with little effort using any of the deep-learning packages. The method performs very well in a series of image classification experiments, achieving adaptation effect in the presence of big domain shifts and outperforming previous state-of-the-art on Office datasets. <s> BIB034 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> We address a challenging unsupervised domain adaptation problem with imbalanced cross-domain data. For standard unsupervised domain adaptation, one typically obtains labeled data in the source domain and only observes unlabeled data in the target domain. However, most existing works do not consider the scenarios in which either the label numbers across domains are different, or the data in the source and/or target domains might be collected from multiple datasets. To address the aforementioned settings of imbalanced cross-domain data, we propose Closest Common Space Learning (CCSL) for associating such data with the capability of preserving label and structural information within and across domains. Experiments on multiple cross-domain visual classification tasks confirm that our method performs favorably against state-of-the-art approaches, especially when imbalanced cross-domain data are presented. <s> BIB035 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Recently, a particular paradigm [18] in the domain adaptation field has received considerable attention by introducing novel and important insights to the problem. In this case, the source and target domains are represented in the form of subspaces, which are treated as points on the Grassmann manifold. The geodesic curve between them is sampled to obtain intermediate points. Then a classifier is learnt using the projections of the data onto these subspaces. Despite its relevance and popularity, this paradigm [18] contains some limitations. Firstly, in real-world applications, that simple curve (i.e. shortest path) does not provide the necessary flexibility to model the domain shift between the training and testing data sets. Secondly, by using the geodesic curve, we are restricted to only one source domain, which does not allow to take fully advantage of the multiple datasets that are available nowadays. It is then, natural to ask whether this popular concept could be extended to deal with more complex curves and to integrate multi-sources domains. This is a hard problem considering the Riemannian structure of the space, but we propose a mathematically well-founded idea that enables us to solve it. We exploit the geometric insight of rolling maps [30] to compute a spline curve on the Grassmann manifold. The benefits of the proposed idea are demonstrated through several empirical studies on standard datasets. This novel concept allows to explicitly integrate multi-source domains while the previous one [18] uses the mean of all sources. This enables to model better the domain shift and take fully advantage of the training datasets. <s> BIB036 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Domain adaptation (DA) tackles the problem where data from the training set (source domain) and test set (target domain) have different underlying distributions. For instance, training and testing images may be acquired under different environments, viewpoints and illumination conditions. In this paper, we focus on the more challenging unsupervised DA problem where the samples in the target domain are unlabeled. It is noticed that dictionary learning has gained a lot of popularity due to the fact that images of interest could be reconstructed sparsely in an appropriately learned dictionary [1]. Specifically, we propose a novel domain-adaptive dictionary learning approach to generate a set of intermediate domains which bridge the gap between source and target domains. Our approach defines two types of dictionaries: a common dictionary and a domain-specific dictionary. The overall learning process illustrated in Figure 1 consists of three steps: (1) At the beginning, we first learn the common dictionary DC, domainspecific dictionaries D0 and Dt for source and target domains. (2) At the k-th step, we enforce the recovered feature representations of target data in all available domains to have the same sparse codes, while adapting the most recently obtained dictionary Dk to better represent the target domain. Then we multiply dictionaries in the k-th domain with the corresponding sparse codes to recover feature representations of target data Xk t in this domain. (3) We update Dk to find the next domain-specific dictionary Dk+1 by further minimizing the reconstruction error in representing the target data. Then we alternate between the steps of sparse coding and dictionary updating until the stopping criteria is satisfied. Notations: Let X s ∈ Rd×Ns , X t ∈ Rd×Nt be the feature representations of source and target data respectively, where d is the feature dimension, Ns and Nt are the number of samples in the two domains. The feature representations of recovered source and target data in the k-th intermediate domain are denoted as Xs ∈ Rd×Ns and Xk t ∈ Rd×Nt respectively. The common dictionary is denoted as DC, whereas source-specific and targetspecific dictionaries are denoted as D0, Dt respectively. Similarly, we use Dk,k = 1...N to denote the domain-specific dictionary for the k-th domain, where N is the number of intermediate domains. We set all the dictionaries to be of the same size ∈Rd×n. At the beginning, we learn the common dictionary DC by minimizing the reconstruction error of both source and target data as follows: <s> BIB037 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Recent researches on transfer learning exploit deep structures for discriminative feature representation to tackle cross-domain disparity. However, few of them are able to joint feature learning and knowledge transfer in a unified deep framework. In this paper, we develop a novel approach, called Deep Low-Rank Coding (DLRC), for transfer learning. Specifically, discriminative low-rank coding is achieved in the guidance of an iterative supervised structure term for each single layer. In this way, both marginal and conditional distributions between two domains intend to be mitigated. In addition, a marginalized denoising feature transformation is employed to guarantee the learned single-layer low-rank coding to be robust despite of corruptions or noises. Finally, by stacking multiple layers of low-rank codings, we manage to learn robust cross-domain features from coarse to fine. Experimental results on several benchmarks have demonstrated the effectiveness of our proposed algorithm on facilitating the recognition performance for the target domain. <s> BIB038 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> In many real-world applications, the domain of model learning (referred as source domain) is usually inconsistent with or even different from the domain of testing (referred as target domain), which makes the learnt model degenerate in target domain, i.e., the test domain. To alleviate the discrepancy between source and target domains, we propose a domain adaptation method, named as Bi-shifting Auto-Encoder network (BAE). The proposed BAE attempts to shift source domain samples to target domain, and also shift the target domain samples to source domain. The non-linear transformation of BAE ensures the feasibility of shifting between domains, and the distribution consistency between the shifted domain and the desirable domain is constrained by sparse reconstruction between them. As a result, the shifted source domain is supervised and follows similar distribution as target domain. Therefore, any supervised method can be applied on the shifted source domain to train a classifier for classification in target domain. The proposed method is evaluated on three domain adaptation scenarios of face recognition, i.e., domain adaptation across view angle, ethnicity, and imaging sensor, and the promising results demonstrate that our proposed BAE can shift samples between domains and thus effectively deal with the domain discrepancy. <s> BIB039 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> We propose a novel method for unsupervised domain adaptation. Traditional machine learning algorithms often fail to generalize to new input distributions, causing reduced accuracy. Domain adaptation attempts to compensate for the performance degradation by transferring and adapting source knowledge to target domain. Existing unsupervised methods project domains into a lower-dimensional space and attempt to align the subspace bases, effectively learning a mapping from source to target points or vice versa. However, they fail to take into account the difference of the two distributions in the subspaces, resulting in misalignment even after adaptation. We present a unified view of existing subspace mapping based methods and develop a generalized approach that also aligns the distributions as well as the subspace bases. We provide a detailed evaluation of our approach on benchmark datasets and show improved results over published approaches. <s> BIB040 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Recent studies reveal that a deep neural network can learn transferable features which generalize well to novel tasks for domain adaptation. However, as deep features eventually transition from general to specific along the network, the feature transferability drops significantly in higher layers with increasing domain discrepancy. Hence, it is important to formally reduce the dataset bias and enhance the transferability in task-specific layers. In this paper, we propose a new Deep Adaptation Network (DAN) architecture, which generalizes deep convolutional neural network to the domain adaptation scenario. In DAN, hidden representations of all task-specific layers are embedded in a reproducing kernel Hilbert space where the mean embeddings of different domain distributions can be explicitly matched. The domain discrepancy is further reduced using an optimal multikernel selection method for mean embedding matching. DAN can learn transferable features with statistical guarantees, and can scale linearly by unbiased estimate of kernel embedding. Extensive empirical evidence shows that the proposed architecture yields state-of-the-art image classification error rates on standard domain adaptation benchmarks. <s> BIB041 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Conventional metric learning methods usually assume that the training and test samples are captured in similar scenarios so that their distributions are assumed to be the same. This assumption does not hold in many real visual recognition applications, especially when samples are captured across different data sets. In this paper, we propose a new deep transfer metric learning (DTML) method to learn a set of hierarchical nonlinear transformations for cross-domain visual recognition by transferring discriminative knowledge from the labeled source domain to the unlabeled target domain. Specifically, our DTML learns a deep metric network by maximizing the inter-class variations and minimizing the intra-class variations, and minimizing the distribution divergence between the source domain and the target domain at the top layer of the network. To better exploit the discriminative information from the source domain, we further develop a deeply supervised transfer metric learning (DSTML) method by including an additional objective on DTML, where the output of both the hidden layers and the top layer are optimized jointly. To preserve the local manifold of input data points in the metric space, we present two new methods, DTML with autoencoder regularization and DSTML with autoencoder regularization. Experimental results on face verification, person re-identification, and handwritten digit recognition validate the effectiveness of the proposed methods. <s> BIB042 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Recent reports suggest that a generic supervised deep CNN model trained on a large-scale dataset reduces, but does not remove, dataset bias. Fine-tuning deep models in a new domain can require a significant amount of labeled data, which for many applications is simply not available. We propose a new CNN architecture to exploit unlabeled and sparsely labeled target domain data. Our approach simultaneously optimizes for domain invariance to facilitate domain transfer and uses a soft label distribution matching loss to transfer information between tasks. Our proposed adaptation method offers empirical performance which exceeds previously published results on two standard benchmark visual domain adaptation tasks, evaluated across supervised and semi-supervised adaptation settings. <s> BIB043 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> We introduce a new representation learning approach for domain adaptation, in which data at training and test time come from similar but different distributions. Our approach is directly inspired by the theory on domain adaptation suggesting that, for effective domain transfer to be achieved, predictions must be made based on features that cannot discriminate between the training (source) and test (target) domains. ::: ::: The approach implements this idea in the context of neural network architectures that are trained on labeled data from the source domain and unlabeled data from the target domain (no labeled target-domain data is necessary). As the training progresses, the approach promotes the emergence of features that are (i) discriminative for the main learning task on the source domain and (ii) indiscriminate with respect to the shift between the domains. We show that this adaptation behaviour can be achieved in almost any feed-forward model by augmenting it with few standard layers and a new gradient reversal layer. The resulting augmented architecture can be trained using standard backpropagation and stochastic gradient descent, and can thus be implemented with little effort using any of the deep learning packages. ::: ::: We demonstrate the success of our approach for two distinct classification problems (document sentiment analysis and image classification), where state-of-the-art domain adaptation performance on standard benchmarks is achieved. We also validate the approach for descriptor learning task in the context of person re-identification application. <s> BIB044 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Domain adaptation (DA) has gained a lot of success in the recent years in computer vision to deal with situations where the learning process has to transfer knowledge from a source to a target domain. In this paper, we introduce a novel unsupervised DA approach based on both subspace alignment and selection of landmarks similarly distributed between the two domains. Those landmarks are selected so as to reduce the discrepancy between the domains and then are used to non linearly project the data in the same space where an efficient subspace alignment (in closed-form) is performed. We carry out a large experimental comparison in visual domain adaptation showing that our new method outperforms the most recent unsupervised DA approaches. <s> BIB045 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> In this paper we consider a version of the zero-shot learning problem where seen class source and target domain data are provided. The goal during test-time is to accurately predict the class label of an unseen target domain instance based on revealed source domain side information (\eg attributes) for unseen classes. Our method is based on viewing each source or target data as a mixture of seen class proportions and we postulate that the mixture patterns have to be similar if the two instances belong to the same unseen class. This perspective leads us to learning source/target embedding functions that map an arbitrary source/target domain data into a same semantic space where similarity can be readily measured. We develop a max-margin framework to learn these similarity functions and jointly optimize parameters by means of cross validation. Our test results are compelling, leading to significant improvement in terms of accuracy on most benchmark datasets for zero-shot recognition. <s> BIB046 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Zero-shot learning (ZSL) can be considered as a special case of transfer learning where the source and target domains have different tasks/label spaces and the target domain is unlabelled, providing little guidance for the knowledge transfer. A ZSL method typically assumes that the two domains share a common semantic representation space, where a visual feature vector extracted from an image/video can be projected/embedded using a projection function. Existing approaches learn the projection function from the source domain and apply it without adaptation to the target domain. They are thus based on naive knowledge transfer and the learned projections are prone to the domain shift problem. In this paper a novel ZSL method is proposed based on unsupervised domain adaptation. Specifically, we formulate a novel regularised sparse coding framework which uses the target domain class labels' projections in the semantic space to regularise the learned target domain projection thus effectively overcoming the projection domain shift problem. Extensive experiments on four object and action recognition benchmark datasets show that the proposed ZSL method significantly outperforms the state-of-the-arts. <s> BIB047 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Most existing person re-identification (Re-ID) approaches follow a supervised learning framework, in which a large number of labelled matching pairs are required for training. This severely limits their scalability in realworld applications. To overcome this limitation, we develop a novel cross-dataset transfer learning approach to learn a discriminative representation. It is unsupervised in the sense that the target dataset is completely unlabelled. Specifically, we present an multi-task dictionary learning method which is able to learn a dataset-shared but targetdata-biased representation. Experimental results on five benchmark datasets demonstrate that the method significantly outperforms the state-of-the-art. <s> BIB048 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Unsupervised domain adaptation (UDA) deals with the task that labeled training and unlabeled test data collected from source and target domains, respectively. In this paper, we particularly address the practical and challenging scenario of imbalanced cross-domain data. That is, we do not assume the label numbers across domains to be the same, and we also allow the data in each domain to be collected from multiple datasets/sub-domains. To solve the above task of imbalanced domain adaptation, we propose a novel algorithm of Domain-constraint Transfer Coding (DcTC). Our DcTC is able to exploit latent subdomains within and across data domains, and learns a common feature space for joint adaptation and classification purposes. Without assuming balanced cross-domain data as most existing UDA approaches do, we show that our method performs favorably against state-of-the-art methods on multiple cross-domain visual classification tasks. <s> BIB049 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> In this paper, we propose a novel unsupervised domain adaptation algorithm based on deep learning for visual object recognition. Specifically, we design a new model called Deep Reconstruction-Classification Network (DRCN), which jointly learns a shared encoding representation for two tasks: (i) supervised classification of labeled source data, and (ii) unsupervised reconstruction of unlabeled target data. In this way, the learnt representation not only preserves discriminability, but also encodes useful information from the target domain. Our new DRCN model can be optimized by using backpropagation similarly as the standard neural networks. <s> BIB050 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> We propose coupled generative adversarial network (CoGAN) for learning a joint distribution of multi-domain images. In contrast to the existing approaches, which require tuples of corresponding images in different domains in the training set, CoGAN can learn a joint distribution without any tuple of corresponding images. It can learn a joint distribution with just samples drawn from the marginal distributions. This is achieved by enforcing a weight-sharing constraint that limits the network capacity and favors a joint distribution solution over a product of marginal distributions one. We apply CoGAN to several joint distribution learning tasks, including learning a joint distribution of color and depth images, and learning a joint distribution of face images with different attributes. For each task it successfully learns the joint distribution without any tuple of corresponding images. We also demonstrate its applications to domain adaptation and image transformation. <s> BIB051 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Deep networks have been successfully applied to learn transferable features for adapting models from a source domain to a different target domain. In this paper, we present joint adaptation networks (JAN), which learn a transfer network by aligning the joint distributions of multiple domain-specific layers across domains based on a joint maximum mean discrepancy (JMMD) criterion. Adversarial training strategy is adopted to maximize JMMD such that the distributions of the source and target domains are made more distinguishable. Learning can be performed by stochastic gradient descent with the gradients computed by back-propagation in linear-time. Experiments testify that our model yields state of the art results on standard datasets. <s> BIB052 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> The recent success of deep neural networks relies on massive amounts of labeled data. For a target task where labeled data is unavailable, domain adaptation can transfer a learner from a different source domain. In this paper, we propose a new approach to domain adaptation in deep networks that can jointly learn adaptive classifiers and transferable features from labeled data in the source domain and unlabeled data in the target domain. We relax a shared-classifier assumption made by previous methods and assume that the source classifier and target classifier differ by a residual function. We enable classifier adaptation by plugging several layers into deep network to explicitly learn the residual function with reference to the target classifier. We fuse features of multiple layers with tensor product and embed them into reproducing kernel Hilbert spaces to match distributions for feature adaptation. The adaptation can be achieved in most feed-forward models by extending them with new residual layers and loss functions, which can be trained efficiently via back-propagation. Empirical evidence shows that the new approach outperforms state of the art methods on standard domain adaptation benchmarks. <s> BIB053 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation. <s> BIB054 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Deep feature learning has recently emerged with demonstrated effectiveness in domain adaptation. In this paper, we propose a Deep Nonlinear Feature Coding framework (DNFC) for unsupervised domain adaptation. DNFC builds on the marginalized stacked denoising autoencoder (mSDA) to extract rich deep features. We introduce two new elements to mSDA: domain divergence minimization by Maximum Mean Discrepancy (MMD), and nonlinear coding by kernelization. These two elements are essential for domain adaptation as they ensure the extracted deep features to have a small distribution discrepancy and encode data nonlinearity. The effectiveness of DNFC is verified by extensive experiments on benchmark datasets. Specifically, DNFC attains much higher prediction accuracy than state-of-the-art domain adaptation methods. Compared to its basis mSDA, DNFC is able to achieve remarkable prediction improvement and meanwhile converges much faster with a small number of stacked layers. <s> BIB055 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> The cost of large scale data collection and annotation often makes the application of machine learning algorithms to new tasks or datasets prohibitively expensive. One approach circumventing this cost is training models on synthetic data where annotations are provided automatically. Despite their appeal, such models often fail to generalize from synthetic to real images, necessitating domain adaptation algorithms to manipulate these models before they can be successfully applied. Existing approaches focus either on mapping representations from one domain to the other, or on learning to extract features that are invariant to the domain from which they were extracted. However, by focusing only on creating a mapping or shared representation between the two domains, they ignore the individual characteristics of each domain. We hypothesize that explicitly modeling what is unique to each domain can improve a model's ability to extract domain-invariant features. Inspired by work on private-shared component analysis, we explicitly learn to extract image representations that are partitioned into two subspaces: one component which is private to each domain and one which is shared across domains. Our model is trained to not only perform the task we care about in the source domain, but also to use the partitioned representation to reconstruct the images from both domains. Our novel architecture results in a model that outperforms the state-of-the-art on a range of unsupervised domain adaptation scenarios and additionally produces visualizations of the private and shared representations enabling interpretation of the domain adaptation process. <s> BIB056 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> State-of-the-art methods for zero-shot visual recognition formulate learning as a joint embedding problem of images and side information. In these formulations the current best complement to visual features are attributes: manuallyencoded vectors describing shared characteristics among categories. Despite good performance, attributes have limitations: (1) finer-grained recognition requires commensurately more attributes, and (2) attributes do not provide a natural language interface. We propose to overcome these limitations by training neural language models from scratch, i.e. without pre-training and only consuming words and characters. Our proposed models train end-to-end to align with the fine-grained and category-specific content of images. Natural language provides a flexible and compact way of encoding only the salient visual aspects for distinguishing categories. By training on raw text, our model can do inference on raw text as well, providing humans a familiar mode both for annotation and retrieval. Our model achieves strong performance on zero-shot text-based image retrieval and significantly outperforms the attribute-based state-of-the-art for zero-shot classification on the Caltech-UCSD Birds 200-2011 dataset. <s> BIB057 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Zero-shot recognition (ZSR) deals with the problem of predicting class labels for target domain instances based on source domain side information (e.g. attributes) of unseen classes. We formulate ZSR as a binary prediction problem. Our resulting classifier is class-independent. It takes an arbitrary pair of source and target domain instances as input and predicts whether or not they come from the same class, i.e. whether there is a match. We model the posterior probability of a match since it is a sufficient statistic and propose a latent probabilistic model in this context. We develop a joint discriminative learning framework based on dictionary learning to jointly learn the parameters of our model for both domains, which ultimately leads to our class-independent classifier. Many of the existing embedding methods can be viewed as special cases of our probabilistic model. On ZSR our method shows 4.90% improvement over the state-of-the-art in accuracy averaged across four benchmark datasets. We also adapt ZSR method for zero-shot retrieval and show 22.45% improvement accordingly in mean average precision (mAP). <s> BIB058 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> We develop a novel method for zero shot learning (ZSL) based on test-time adaptation of similarity functions learned using training data. Existing methods exclusively employ source-domain side information for recognizing unseen classes during test time. We show that for batch-mode applications, accuracy can be significantly improved by adapting these predictors to the observed test-time target-domain ensemble. We develop a novel structured prediction method for maximum a posteriori (MAP) estimation, where parameters account for test-time domain shift from what is predicted primarily using source domain information. We propose a Gaussian parameterization for the MAP problem and derive an efficient structure prediction algorithm. Empirically we test our method on four popular benchmark image datasets for ZSL, and show significant improvement over the state-of-the-art, on average, by 11.50 % and 30.12 % in terms of accuracy for recognition and mean average precision (mAP) for retrieval, respectively. <s> BIB059 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Domain adaptation from one data space (or domain) to the other is one of the most challenging tasks of modern data analytics. If the adaptation is done cor-rectly, models built on a specific data space become able to process data depicting the same semantic concepts (the classes), but observed by another observation system with its own specificities. In this paper, we propose an optimal transporta-tion model that aligns the representations in the source and target domains. We learn a transportation plan matching both PDFs and constrain labeled samples in the source domain to remain close during transport with a non convex group regularization. This way, we exploit at the same time the labeled information in the source (the same that will be used by the classifier after adaptation) and the unlabeled distributions observed in both domains. We propose an efficient majoration-minimization algorithm to solve the resultaing optimization problem and discuss its convergence. Numerical experiments of real data show the interest of the method, that outperforms state-of-the-art approaches. <s> BIB060 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> This paper presents a novel unsupervised domain adaptation method for cross-domain visual recognition. We propose a unified framework that reduces the shift between domains both statistically and geometrically, referred to as Joint Geometrical and Statistical Alignment (JGSA). Specifically, we learn two coupled projections that project the source domain and target domain data into low-dimensional subspaces where the geometrical shift and distribution shift are reduced simultaneously. The objective function can be solved efficiently in a closed form. Extensive experiments have verified that the proposed method significantly outperforms several state-of-the-art domain adaptation methods on a synthetic dataset and three different real world cross-domain visual recognition tasks. <s> BIB061 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> The learning of domain-invariant representations in the context of domain adaptation with neural networks is considered. We propose a new regularization method that minimizes the discrepancy between domain-specific latent feature representations directly in the hidden activation space. Although some standard distribution matching approaches exist that can be interpreted as the matching of weighted sums of moments, e.g. Maximum Mean Discrepancy (MMD), an explicit order-wise matching of higher order moments has not been considered before. We propose to match the higher order central moments of probability distributions by means of order-wise moment differences. Our model does not require computationally expensive distance and kernel matrix computations. We utilize the equivalent representation of probability distributions by moment sequences to define a new distance function, called Central Moment Discrepancy (CMD). We prove that CMD is a metric on the set of probability distributions on a compact interval. We further prove that convergence of probability distributions on compact intervals w.r.t. the new metric implies convergence in distribution of the respective random variables. We test our approach on two different benchmark data sets for object recognition (Office) and sentiment analysis of product reviews (Amazon reviews). CMD achieves a new state-of-the-art performance on most domain adaptation tasks of Office and outperforms networks trained with MMD, Variational Fair Autoencoders and Domain Adversarial Neural Networks on Amazon reviews. In addition, a post-hoc parameter sensitivity analysis shows that the new approach is stable w.r.t. parameter changes in a certain interval. The source code of the experiments is publicly available. <s> BIB062 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Metric learning has attracted increasing attention due to its critical role in image analysis and classification. Conventional metric learning always assumes that the training and test data are sampled from the same or similar distribution. However, to build an effective distance metric, we need abundant supervised knowledge (i.e., side/label information), which is generally inaccessible in practice, because of the expensive labeling cost. In this paper, we develop a robust transfer metric learning (RTML) framework to effectively assist the unlabeled target learning by transferring the knowledge from the well-labeled source domain. Specifically, RTML exploits knowledge transfer to mitigate the domain shift in two directions, i.e., sample space and feature space. In the sample space, domain-wise and class-wise adaption schemes are adopted to bridge the gap of marginal and conditional distribution disparities across two domains. In the feature space, our metric is built in a marginalized denoising fashion and low-rank constraint, which make it more robust to tackle noisy data in reality. Furthermore, we design an explicit rank constraint regularizer to replace the rank minimization NP-hard problem to guide the low-rank metric learning. Experimental results on several standard benchmarks demonstrate the effectiveness of our proposed RTML by comparing it with the state-of-the-art transfer learning and metric learning algorithms. <s> BIB063 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Adversarial learning methods are a promising approach to training robust deep networks, and can generate complex samples across diverse domains. They can also improve recognition despite the presence of domain shift or dataset bias: recent adversarial approaches to unsupervised domain adaptation reduce the difference between the training and test domain distributions and thus improve generalization performance. However, while generative adversarial networks (GANs) show compelling visualizations, they are not optimal on discriminative tasks and can be limited to smaller shifts. On the other hand, discriminative approaches can handle larger domain shifts, but impose tied weights on the model and do not exploit a GAN-based loss. In this work, we first outline a novel generalized framework for adversarial adaptation, which subsumes recent state-of-the-art approaches as special cases, and use this generalized view to better relate prior approaches. We then propose a previously unexplored instance of our general framework which combines discriminative modeling, untied weight sharing, and a GAN loss, which we call Adversarial Discriminative Domain Adaptation (ADDA). We show that ADDA is more effective yet considerably simpler than competing domain-adversarial methods, and demonstrate the promise of our approach by exceeding state-of-the-art unsupervised adaptation results on standard domain adaptation tasks as well as a difficult cross-modality object classification task. <s> BIB064 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Collecting well-annotated image datasets to train modern machine learning algorithms is prohibitively expensive for many tasks. One appealing alternative is rendering synthetic data where ground-truth annotations are generated automatically. Unfortunately, models trained purely on rendered images fail to generalize to real images. To address this shortcoming, prior work introduced unsupervised domain adaptation algorithms that have tried to either map representations between the two domains, or learn to extract features that are domain-invariant. In this work, we approach the problem in a new light by learning in an unsupervised manner a transformation in the pixel space from one domain to the other. Our generative adversarial network (GAN)-based method adapts source-domain images to appear as if drawn from the target domain. Our approach not only produces plausible samples, but also outperforms the state-of-the-art on a number of unsupervised domain adaptation scenarios by large margins. Finally, we demonstrate that the adaptation process generalizes to object classes unseen during training. <s> BIB065 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> It is important to apply models trained on a large number of labeled samples to different domains because collecting many labeled samples in various domains is expensive. To learn discriminative representations for the target domain, we assume that artificially labeling the target samples can result in a good representation. Tri-training leverages three classifiers equally to provide pseudo-labels to unlabeled samples; however, the method does not assume labeling samples generated from a different domain. In this paper, we propose the use of an asymmetric tri-training method for unsupervised domain adaptation, where we assign pseudo-labels to unlabeled samples and train the neural networks as if they are true labels. In our work, we use three networks asymmetrically, and by asymmetric, we mean that two networks are used to label unlabeled target samples, and one network is trained by the pseudo-labeled samples to obtain target-discriminative representations. Our proposed method was shown to achieve a state-of-the-art performance on the benchmark digit recognition datasets for domain adaptation. <s> BIB066 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Unlabelled Target Dataset <s> Domain Adaptation is an actively researched problem in Computer Vision. In this work, we propose an approach that leverages unsupervised data to bring the source and target distributions closer in a learned joint feature space. We accomplish this by inducing a symbiotic relationship between the learned embedding and a generative adversarial network. This is in contrast to methods which use the adversarial framework for realistic data generation and retraining deep models with such data. We demonstrate the strength and generality of our approach by performing experiments on three different tasks with varying levels of difficulty: (1) Digit classification (MNIST, SVHN and USPS datasets) (2) Object recognition using OFFICE dataset and (3) Domain adaptation from synthetic to real data. Our method achieves state-of-the art performance in most experimental settings and by far the only GAN-based method that has been shown to work well across different datasets such as OFFICE and DIGITS. <s> BIB067
In this problem, no labelled data in target domain are available but sufficient unlabelled target domain data are observable in the training stage. Table III gives characteristics of problem 3.3, where source domain has sufficient labelled data while target domain has sufficient unlabelled data. This problem is also named unsupervised domain adaptation. The unsupervised domain adaptation has been attracted increasing attention in recent years, which is supposed to be more challenging and realistic. Instance-based Transfer. BIB001 present the kernel mean matching (KMM) method which directly produces resampling weights using MMD criterion, without the need of explicit distribution estimation. BIB010 propose a twostage domain adaptation methodology which combines weighted data from multiple sources based on marginal probability differences (first stage) using MMD criterion as well as conditional probability differences (second stage) by computing the weights of multiple sources.  select a subset of the source data by choosing the samples that are distributed most similarly to the target using MMD criterion. A balancing constraint is added to balance the selected number of landmarks in each class. Differently, BIB004 propose to estimate the sample importance using Kullback-Leibler divergence without explicit density estimation. Rather than assuming single domain in a dataset, some methods assume a dataset may contain several distinctive sub-domains due to the large variations in visual data. For example, BIB017 automatically discover latent domains from multiple sources. The latent domains are characteristic in terms of their large inter-domain variations and can derive strong discriminative models to apply to new testing data. Feature Representation-based Transfer. The first line of feature representationbased methods use the MMD criterion as a distance measure for two distributions. propose a kernel learning method to minimizes the distance between distributions of the data. In addition to the idea in BIB005 minimize the distribution shift, while at the same time minimizing the empirical loss on the labelled data in the source domain. Pan et al. [2009] then proposed transfer component analysis (TCA), for domain adaptation by extending kernel learning in ] to learn a parametric kernel map, which can deal with out-of-sample problem and is more efficient in solving the objective function. BIB018 compare the distributions on the transformed data in RKHS space rather than a lower-dimensional space as in TCA. Rather than merely minimizing marginal distribution alone, adapt the joint distribution in a principled dimensionality reduction procedure by iteratively estimating pseudo labels of target data for the adaptation of conditional distribution. BIB011 propose a domain adaptation metric learning (DAML), by introducing the MMD regularization between domains to the conventional metric learning framework. Instead of using MMD to compare two distributions, other statistic criteria, such as Hellinger distance on statistical manifolds, Quadratic divergence, and mutual information, are also used in this settings among literature BIB027 BIB009 BIB015 . BIB027 argued that MMD criterion does not exploit the fact that probability distributions lie on a Riemannian manifold. They proposed to make better use of the structure of the manifold and rely on Hellinger distance on the manifold to compare the source and target distributions. In the work of BIB009 , the Quadratic divergence-based regularization is used to measure the difference between distributions. Thus, a family of subspace learning algorithms can be used for reduce the domain shift by adding the Bregman divergence-based regularization. reduce the distribution divergence from the information-theoretic perspective, where the joint distribution divergence is reduced by minimizing the mutual information between domains maximizing the mutual information between the data and the estimated labels based on the discriminative clustering assumption. propose the CORrelation ALignment (CORAL) to minimize domain shift by aligning the second-order statistics. Different from previous global transformation, Optimal Transport BIB060 ] defines a local transformation for each sample in the source domain. The domain adaptation problem is seen as a graph matching problem, where each source sample are mapped on target samples under the constraint of marginal distribution preservation. Classifier-based Transfer. Rather than reducing the distribution shift in the feature space, some methods use MMD criteria to regularize the classifier trained on the source domain using target domain unlabelled data. For instance, BIB006 incorporate the MMD criterion as a constraint into the SVM paradigm. Hence, it achieves a trade-off between the large margin class separation in the source domain and the minimization of marginal distribution discrepancy between domains, as projected along the linear classifier. Similarly, extend Quanz's method to reduce both marginal and conditional distribution discrepancies between domains in the domain invariant classifier learning framework. Hybrid Knowledge Transfer. Some methods jointly re-weight instances and map feature to a common space BIB035 . reduce the domain divergence by jointly matching the features and re-weighting the source domain instances in a low-dimensional space using MMD. There are also methods assume that the target domain contains a subset of categories presented in the source dataset. For example, BIB035 propose Closest Common Space Learning (CCSL) for associating data that the label numbers across domains are different or collected from multiple datasets, with the capability of preserving label and structural information within and across domains. Feature Representation-based Transfer. BIB012 propose a Sampling Geodesic Flow (SGF) method by creating intermediate representations of data between the two domains. The generative subspaces created from these intermediate domains are viewed as points on the Grassmann manifold, and points along the geodesic between them are sampled to obtain subspaces for adapting between domains. However, the limitation is that the number of intermediate points is a hard-to-determine hyperparameter. BIB016 tackle this problem by extending the idea in BIB012 ] to a geodesic flow kernel (GFK) method that integrates an infinite number of subspaces that characterize changes from the source to the target domain. The methods in BIB012 BIB028 and BIB016 BIB029 open the opportunity of researches on constructing intermediate domains to bridge the mismatch. For example, BIB036 extend SGF BIB012 method to multiple source datasets. They use smooth polynomial functions described by splines on the manifold to interpolate between all the source domains and the target domain. BIB019 connect the source and target domain by interpolate virtual views by a virtual path for the cross-view action recognition. Rather than manipulating on the subspaces, BIB030 characterize samples from each domain as one covariance matrix and interpolate some intermediate points (i.e., covariance matrices) to bridge the two domains. There are also methods BIB020 BIB037 ] generate a set of intermediate dictionaries by learning the domain-adaptive dictionaries between domains. Instead of modelling intermediate domains, some methods align the two domains directly BIB021 BIB031 ]. For instance, BIB021 learn a mapping function which aligns the source subspace with the target one directly. BIB031 propose to align the manifolds defined by source and target datasets by integrating geometry structure matching, feature matching and geometry preserving. Most of the existing methods assume a global shift, but ignore the individual class differences across domains. take the individual class differences into consideration and generate one joint subspace for each class independently. Then assign labels to the anchor subspaces (with unlabelled samples that are close to each other) in the target domain for guiding the learning of joint subspace of each class. Specifically, the labels of anchor subspaces are assigned using the principle angles as a measure for calculating the similarities between them and source subspaces. Feature Representation-based Transfer. BIB002 propose a structural correspondence learning (SCL) method to identify correspondences among features from different domains. The correlations are modelled with pivot features that behave in the same way for discriminative learning in both domains. The extracted pivot features are used to learn a mapping that maps the original feature of both domains to a shared feature space. The low-rank criterion is also used to guide the learning of domain invariant representations BIB015 BIB038 . BIB015 propose to capture the intrinsic relatedness of the source samples using a low-rank structure and meanwhile identifies the noise and outlier information using a sparse structure. Specifically, the source samples are transformed to an intermediate representation such that each source sample can be linearly represented by the samples of target domain. Similarly, propose to project both source and target data to a generalized subspace where each target sample can be represented by a low-rank transformation of source samples. BIB038 extend the low-rank coding method to a deep low-rank coding method, where multiple layers are stacked to learn the multi-level features across two domains. Each layer is constrained by a low-rank coding. There are also methods use dictionary learning method to learn domain invariant representations BIB048 BIB049 . BIB048 propose a Multitask Dictionary Learning (UMDL) model to learn view-invariant and discriminative information for the task of person re-identification. Three types of dictionaries are learnt: task shared dictionary that is dataset invariant, target specific dictionary that is view-invariant, and task specific residual dictionary that encodes the residual parts of features that cannot be modelled by the source task. Similar to BIB035 , BIB049 also assume that the target domain contains a subset of categories presented in the source dataset. They derive a domain invariant space for aligning and representing cross domain data with a locality constraint sparse coding method. Bengio [2012] argue that deep neural networks can learn more transferable features by disentangling the unknown factors of variation that underlie the training distribution. BIB022 propose the deep convolutional representations named DeCAF, where a deep convolutional neural network is pre-trained using the previous large scale source dataset in a fully supervised fashion. Then transfer the features (defined by the convolutional network weights learned on source dataset) to the target data. The deep auto-encoders are also used for the cross-dataset tasks BIB013 BIB039  BIB050 . BIB013 propose to use stacked denoising autoencoders (SDAs) to learn more transferable features for domain adaptation. BIB039 propose a Bi-shifting Auto-Encoder Network (BAE), which has one common encoder, two domain specific decoders. The proposed BAE can shift source domain samples to the target domain and also shift the target domain data to the source domain using the sparse reconstruction to ensure the distribution consistency. BIB050 propose a Deep Reconstruction-Classification Network (DRCN), which jointly learns a shared encoding representation for two tasks: i) supervised classification of labelled source data, and ii) unsupervised reconstruction of unlabelled target data. identify that the SDAs are limited by their high computational cost. Hence, they propose marginalized SDA (mSDA) by adopting the greedy layer-by-layer training of SDAs for domain adaptation. The CoGAN BIB051 approach applied Generative Adversarial Networks (GANs) BIB032 ] to the domain transfer problem by training two GANs to generate the source and target images respectively. This weight-sharing constraint allows CoGAN to learn a joint distribution of two domains. The intuition is that the images from different domains share the same high-level abstraction but have different low-level realizations. Feature Representation-based Transfer.  propose to learn two dictionaries simultaneously for pairs of correspondence video and encourage each video in the pair to have the same sparse representation, where both correspondence and higher level representation criteria are used. Similarly, BIB023 present a joint model which learns a pair of dictionaries with a feature space for describing and associating cross-domain data. incorporate MMD criterion into the objective function of sparse coding to make the new representations robust to the distribution difference. BIB040 extend the subspace alignment BIB021 ] by aligning the second-order of statistics (covariance) as well as the subspace bases. BIB061 propose to learn two coupled projections that project the source domain and target domain data into low-dimensional subspaces where the geometrical shift and distribution shift are reduced simultaneously using both geometric and statistic criteria. As mentioned in Section3.3.3, deep neural networks can learn more transferable features for domain adaptation  BIB022 , by disentangling explanatory factors of variations underlying data samples, and grouping deep features hierarchically according to their relatedness to invariant factors. However, the features computed in higher layers of the network must depend greatly on the specific dataset and task BIB033 , which are task-specific features and are not safely transferable to novel tasks. Hence, some recent work impose statistic criterion into the deep learning framework to further reduce domain bias. For instance, the MMD criterion is used to reduce divergence of marginal distributions BIB041 or joint distributions BIB052 ] between domains. BIB053 use MMD to learn transferable features and adaptive classifiers. They relax a shared-classifier assumption made by previous methods and assume that the source classifier and target classifier differ by a residual function BIB054 . extend the CORrelation ALignment (CORAL) method ] to align the second-order statistics of the source and target distributions in deep learning framework. BIB062 propose the Central Moment Discrepancy (CMD) method to match the higher order central moments of probability distributions by means of order-wise moment differences, which does not require computationally expensive distance and kernel matrix computations. The statistic criterion is also incorperated into the deep autoencoder. BIB055 propose to use MMD distance to quantify the domain divergence and incorporate it in the learning of the transformation in the mSDA. The Domain Separation Networks (DSN) BIB056 introduces the notion of a private subspace for each domain, which captures domain specific properties, such as background and low level image statistics. A shared subspace, enforced through the use of autoencoders and explicit loss functions (i.e. MMD or H divergence), captures shared features between the domains. The model integrates a reconstruction loss using a shared decoder, which learns to reconstruct the input sample by using both the private and shared representations. BIB042 propose a deep transfer metric learning (DTML) in a deep learning framework, where inter-class variations are maximized and the intra-class variations are minimized, and the distribution divergence between the source domain and the target domain at the top layer of the network is minimized, simultaneously. Similar to BIB042 , BIB063 also propose a metric learning based method. However, their method is developed in marginalized denoising scheme and low-rank constraint is incorporated to guide the cross-domain metric learning by uncovering more common feature structure across two domains. In addition, the marginal and conditional distribution differences across two domains are both leveraged. Inspired by the recent adversarial learning BIB032 , the H divergence is used to encourage samples from different domains to be non-discriminative with respect to domain labels BIB043 BIB034 BIB044 BIB064 BIB065 . For example, BIB043 proposed adding a domain classifier that predicts the binary domain label of the inputs and designed a domain confusion loss to encourage its prediction to be as close as possible to a uniform distribution over binary labels. The gradient reversal algorithm (ReverseGrad) proposed by BIB034 also treats domain invariance as a binary classification problem, but directly maximizes the loss of the domain classifier by reversing its gradients. BIB065 propose a GAN-based method that adapts source-domain images to appear as if drawn from the target domain in pixel level. BIB067 propose an adversarial image generation frame-work to directly learn the shared feature embedding using labelled data from source and unlabelled data from the target. Classifier-based Transfer. BIB003 use both self labelling and statistic criterion to transfer Naive Bayes classifier. They first estimate the initial probabilities under source domain distribution, and then use an EM algorithm to revise the model for the target distribution. Moreover, the Kullback-Leibler (KL) divergence is used to measure the distance between the training and test data. Then the distance is used to estimate the trade-off parameters. BIB066 propose an asymmetric tri-training method for unsupervised domain adaptation, where the pseudo-labels are assigned to unlabelled samples and train the deep neural networks as if they are true labels. Hence, both self-labelling and high level representations are used. Hybrid Knowledge Transfer. BIB045 consider both subspace alignment and selection of landmarks similarly distributed between the two domains. Those landmarks are selected so as to reduce the discrepancy between the domains and then are used to non linearly project the data in the same space. Hence, both geometric and statistic criteria are used for transferring instance and feature representations. This problem assumes no labelled target data is available (see Table X ). We name this problem as unsupervised heterogeneous domain adaptation. In this problem, the feature spaces can be completely different between dataset. On the other hand, it can also be assumed that the source data consist of multiple modalities while the target data only contain one of the modalities, or vice versa. This scenario is also considered under this problem because the relationships between different feature spaces still need to be exploited. Hybrid Knowledge Transfer. assume the source datasets contain multiple modalities and target dataset only contains one modality. However, the target data are also available in the training stage. In addition, they assume the source and target data are from different distributions. Hence, the distribution mismatch also needs to be reduced. Both the feature representation and classifier parameters are transferred by employing both original and privileged knowledge in the source dataset. Specifically, they use the statistic criterion (MMD) to transform the source and target common modality into a shared subspace, in the meantime, the multiple source modalities are also transformed to the same representation in the common space. They iteratively learn the common space and the robust classifier based on the intuition that a suitable common space will be beneficial for learning a more robust classifier; and the robust classifier can also help finding a more discriminative common space. Some research also try to tackle the heterogeneous label space problem by assuming that no labelled data are presented. This problem can be named as unsupervised transfer learning. Feature Representation-based Transfer. The higher-level representation criterion is generally used for this problem. Two different scenarios are considered among the literature. The first scenario is that only the label spaces between source and target datasets are different. Since there are no labels in the target training set, the unseen class information is generally gained from a higher level semantic space shared between datasets. Some research uses Web search the semantic representation linking the different label spaces BIB007 BIB014 . For example, BIB007 assume that there is no labels in the target data, but the number of activities and names of activities are known and the source activities and the target activities have some kind of relationship. They bridge between the activities in two domains by learning a similarity function in the text feature space (word vectors) via Web search. Then they train a weighted SVM model with different probabilistic confidence weights learned from the similarity function. Some other research assumes the shared humanspecified high-level semantic space (i.e. attributes BIB008 ], or text descriptions BIB057 ) between datasets. Given a defined attribute or text description ontology, each class can be represented by a vector in the semantic space. However, the attribute annotations or text descriptions are expansive to acquire. The attributes are substituted by the visual similarity and data distribution information in transductive settings BIB046 BIB058 BIB059 , where the target domain data of unseen classes are required in the training stage to learn the model. Another strategy learns the semantic space by borrowing the large and unrestricted, but freely available, text corpora (i.e. Wikipedia) to derive a word vector space BIB024 BIB025 BIB026 . The related work on semantic space (i.e. attributes, text descriptions, or word vector) will be further discussed in Section 5.4, since the target data are generally not required when the semantic space is involved. The second scenario assumes that in addition to the different label spaces, the domain shift (i.e. the distribution shift of features) also exists between datasets BIB047 . For example, name this problem as projection domain shift and propose a transductive multi-view embedding space that rectifies the projection shift. BIB047 propose a regularised sparse coding framework which uses the target domain class labels' projections in the semantic space to regularise the learned target domain projection thus overcoming the projection domain shift problem.