Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> Learning visual models of object categories notoriously requires hundreds or thousands of training examples. We show that it is possible to learn much information about a category from just one, or a handful, of images. The key insight is that, rather than learning from scratch, one can take advantage of knowledge coming from previously learned categories, no matter how different these categories might be. We explore a Bayesian implementation of this idea. Object categories are represented by probabilistic models. Prior knowledge is represented as a probability density function on the parameters of these models. The posterior model for an object category is obtained by updating the prior in the light of one or more observations. We test a simple implementation of our algorithm on a database of 101 diverse object categories. We compare category models learned by an implementation of our Bayesian approach to models learned from by maximum likelihood (ML) and maximum a posteriori (MAP) methods. We find that on a database of more than 100 categories, the Bayesian approach produces informative models when the number of training examples is too small for other methods to operate successfully. <s> BIB001 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> We present a new machine learning framework called "self-taught learning" for using unlabeled data in supervised classification tasks. We do not assume that the unlabeled data follows the same class labels or generative distribution as the labeled data. Thus, we would like to use a large number of unlabeled images (or audio samples, or text documents) randomly downloaded from the Internet to improve performance on a given image (or audio, or text) classification task. Such unlabeled data is significantly easier to obtain than in typical semi-supervised or transfer learning settings, making self-taught learning widely applicable to many practical learning problems. We describe an approach to self-taught learning that uses sparse coding to construct higher-level features using the unlabeled data. These features form a succinct input representation and significantly improve classification performance. When using an SVM for classification, we further show how a Fisher kernel can be learned for this representation. <s> BIB002 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> We describe an approach to domain adaptation that is appropriate exactly in the case when one has enough ``target'' data to do slightly better than just using only ``source'' data. Our approach is incredibly simple, easy to implement as a preprocessing step (10 lines of Perl!) and outperforms state-of-the-art approaches on a range of datasets. Moreover, it is trivially extended to a multi-domain adaptation problem, where one has data from a variety of different domains. <s> BIB003 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> Many multimedia applications can benefit from techniques for adapting existing classifiers to data with different distributions. One example is cross-domain video concept detection which aims to adapt concept classifiers across various video domains. In this paper, we explore two key problems for classifier adaptation: (1) how to transform existing classifier(s) into an effective classifier for a new dataset that only has a limited number of labeled examples, and (2) how to select the best existing classifier(s) for adaptation. For the first problem, we propose Adaptive Support Vector Machines (A-SVMs) as a general method to adapt one or more existing classifiers of any type to the new dataset. It aims to learn the "delta function" between the original and adapted classifier using an objective function similar to SVMs. For the second problem, we estimate the performance of each existing classifier on the sparsely-labeled new dataset by analyzing its score distribution and other meta features, and select the classifiers with the best estimated performance. The proposed method outperforms several baseline and competing methods in terms of classification accuracy and efficiency in cross-domain concept detection in the TRECVID corpus. <s> BIB004 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> A basic assumption in traditional machine learning is that the training and test data distributions should be identical. This assumption may not hold in many situations in practice, but we may be forced to rely on a different-distribution data to learn a prediction model. For example, this may be the case when it is expensive to label the data in a domain of interest, although in a related but different domain there may be plenty of labeled data available. In this paper, we propose a novel transfer-learning algorithm for text classification based on an EM-based Naive Bayes classifiers. Our solution is to first estimate the initial probabilities under a distribution Dl of one labeled data set, and then use an EM algorithm to revise the model for a different distribution Du of the test data which are unlabeled. We show that our algorithm is very effective in several different pairs of domains, where the distances between the different distributions are measured using the Kullback-Leibler (KL) divergence. Moreover, KL-divergence is used to decide the trade-off parameters in our algorithm. In the experiment, our algorithm outperforms the traditional supervised and semi-supervised learning algorithms when the distributions of the training and test sets are increasingly different. <s> BIB005 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> Traditional machine learning makes a basic assumption: the training and test data should be under the same distribution. However, in many cases, this identical-distribution assumption does not hold. The assumption might be violated when a task from one new domain comes, while there are only labeled data from a similar old domain. Labeling the new data can be costly and it would also be a waste to throw away all the old data. In this paper, we present a novel transfer learning framework called TrAdaBoost, which extends boosting-based learning algorithms (Freund & Schapire, 1997). TrAdaBoost allows users to utilize a small amount of newly labeled data to leverage the old data to construct a high-quality classification model for the new data. We show that this method can allow us to learn an accurate model using only a tiny amount of new data and a large amount of old data, even when the new data are not sufficient to train a model alone. We show that TrAdaBoost allows knowledge to be effectively transferred from the old data to the new. The effectiveness of our algorithm is analyzed theoretically and empirically to show that our iterative algorithm can converge well to an accurate model. <s> BIB006 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> A situation where training and test samples follow different input distributions is called covariate shift. Under covariate shift, standard learning methods such as maximum likelihood estimation are no longer consistent—weighted variants according to the ratio of test and training input densities are consistent. Therefore, accurately estimating the density ratio, called the importance, is one of the key issues in covariate shift adaptation. A naive approach to this task is to first estimate training and test input densities separately and then estimate the importance by taking the ratio of the estimated densities. However, this naive approach tends to perform poorly since density estimation is a hard task particularly in high dimensional cases. In this paper, we propose a direct importance estimation method that does not involve density estimation. Our method is equipped with a natural cross validation procedure and hence tuning parameters such as the kernel width can be objectively optimized. Simulations illustrate the usefulness of our approach. <s> BIB007 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> CCA can be seen as a multiview extension of PCA, in which information from two sources is used for learning by finding a subspace in which the two views are most correlated. However PCA, and by extension CCA, does not use label information. Fisher Discriminant Analysis uses label information to find informative projections, which can be more informative in supervised learning settings. We show that FDA and its dual can both be formulated as generalized eigenproblems, enabling a kernel formulation. We derive a regularised two-view equivalent of Fisher Discriminant Analysis and its corresponding dual, both of which can also be formulated as generalized eigenproblems. We then show that these can be cast as equivalent disciplined convex optimisation problems, and subsequently extended to multiple views. We show experimental results on an EEG dataset and part of the PASCAL 2007 VOC challenge dataset. <s> BIB008 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> Discriminative learning methods for classification perform well when training and test data are drawn from the same distribution. Often, however, we have plentiful labeled training data from a source domain but wish to learn a classifier which performs well on a target domain with a different distribution and little or no labeled training data. In this work we investigate two questions. First, under what conditions can a classifier trained from source data be expected to perform well on target data? Second, given a small amount of labeled target data, how should we combine it during training with the large amount of labeled source data to achieve the lowest target error at test time?We address the first question by bounding a classifier’s target error in terms of its source error and the divergence between the two domains. We give a classifier-induced divergence measure that can be estimated from finite, unlabeled samples from the domains. Under the assumption that there exists some hypothesis that performs well in both domains, we show that this quantity together with the empirical source error characterize the target error of a source-trained classifier.We answer the second question by bounding the target error of a model which minimizes a convex combination of the empirical source and target errors. Previous theoretical work has considered minimizing just the source error, just the target error, or weighting instances from the two domains equally. We show how to choose the optimal combination of source and target error as a function of the divergence, the sample sizes of both domains, and the complexity of the hypothesis class. The resulting bound generalizes the previously studied cases and is always at least as tight as a bound which considers minimizing only the target error or an equal weighting of source and target errors. <s> BIB009 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> We study the problem of object classification when training and test classes are disjoint, i.e. no training examples of the target classes are available. This setup has hardly been studied in computer vision research, but it is the rule rather than the exception, because the world contains tens of thousands of different object classes and for only a very few of them image, collections have been formed and annotated with suitable class labels. In this paper, we tackle the problem by introducing attribute-based classification. It performs object detection based on a human-specified high-level description of the target objects instead of training images. The description consists of arbitrary semantic attributes, like shape, color or even geographic information. Because such properties transcend the specific learning task at hand, they can be pre-learned, e.g. from image datasets unrelated to the current task. Afterwards, new classes can be detected based on their attribute representation, without the need for a new training phase. In order to evaluate our method and to facilitate research in this area, we have assembled a new large-scale dataset, “Animals with Attributes”, of over 30,000 animal images that match the 50 classes in Osherson's classic table of how strongly humans associate 85 semantic attributes with animal classes. Our experiments show that by using an attribute layer it is indeed possible to build a learning object detection system that does not require any training images of the target classes. <s> BIB010 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> The regularization principals [31] lead approximation schemes to deal with various learning problems, e.g., the regularization of the norm in a reproducing kernel Hilbert space for the ill-posed problem. In this paper, we present a family of subspace learning algorithms based on a new form of regularization, which transfers the knowledge gained in training samples to testing samples. In particular, the new regularization minimizes the Bregman divergence between the distribution of training samples and that of testing samples in the selected subspace, so it boosts the performance when training and testing samples are not independent and identically distributed. To test the effectiveness of the proposed regularization, we introduce it to popular subspace learning algorithms, e.g., principal components analysis (PCA) for cross-domain face modeling; and Fisher's linear discriminant analysis (FLDA), locality preserving projections (LPP), marginal Fisher's analysis (MFA), and discriminative locality alignment (DLA) for cross-domain face recognition and text categorization. Finally, we present experimental evidence on both face image data sets and text data sets, suggesting that the proposed Bregman divergence-based regularization is effective to deal with cross-domain learning problems. <s> BIB011 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> We explore an original strategy for building deep networks, based on stacking layers of denoising autoencoders which are trained locally to denoise corrupted versions of their inputs. The resulting algorithm is a straightforward variation on the stacking of ordinary autoencoders. It is however shown on a benchmark of classification problems to yield significantly lower classification error, thus bridging the performance gap with deep belief networks (DBN), and in several cases surpassing it. Higher level representations learnt in this purely unsupervised fashion also help boost the performance of subsequent SVM classifiers. Qualitative experiments show that, contrary to ordinary autoencoders, denoising autoencoders are able to learn Gabor-like edge detectors from natural image patches and larger stroke detectors from digit images. This work clearly establishes the value of using a denoising criterion as a tractable unsupervised objective to guide the learning of useful higher level representations. <s> BIB012 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> Effectively utilizing readily available auxiliary data to improve predictive performance on new modeling tasks is a key problem in data mining. In this research the goal is to transfer knowledge between sources of data, particularly when ground truth information for the new modeling task is scarce or is expensive to collect where leveraging any auxiliary sources of data becomes a necessity. Towards seamless knowledge transfer among tasks, effective representation of the data is a critical but yet not fully explored research area for the data engineer and data miner. Here we present a technique based on the idea of sparse coding, which essentially attempts to find an embedding for the data by assigning feature values based on subspace cluster membership. We modify the idea of sparse coding by focusing the identification of shared clusters between data when source and target data may have different distributions. In our paper, we point out cases where a direct application of sparse coding will lead to a failure of knowledge transfer. We then present the details of our extension to sparse coding, by incorporating distribution distance estimates for the embedded data, and show that the proposed algorithm can overcome the shortcomings of the sparse coding algorithm on synthetic data and achieve improved predictive performance on a real world chemical toxicity transfer learning task. <s> BIB013 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> Adapting the classifier trained on a source domain to recognize instances from a new target domain is an important problem that is receiving recent attention. In this paper, we present one of the first studies on unsupervised domain adaptation in the context of object recognition, where we have labeled data only from the source domain (and therefore do not have correspondences between object categories across domains). Motivated by incremental learning, we create intermediate representations of data between the two domains by viewing the generative subspaces (of same dimension) created from these domains as points on the Grassmann manifold, and sampling points along the geodesic between them to obtain subspaces that provide a meaningful description of the underlying domain shift. We then obtain the projections of labeled source domain data onto these subspaces, from which a discriminative classifier is learnt to classify projected data from the target domain. We discuss extensions of our approach for semi-supervised adaptation, and for cases with multiple source and target domains, and report competitive results on standard datasets. <s> BIB014 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> The exponential increase in the availability of online reviews and recommendations makes sentiment classification an interesting topic in academic and industrial research. Reviews can span so many different domains that it is difficult to gather annotated training data for all of them. Hence, this paper studies the problem of domain adaptation for sentiment classifiers, hereby a system is trained on labeled reviews from one source domain but is meant to be deployed on another. We propose a deep learning approach which learns to extract a meaningful representation for each review in an unsupervised fashion. Sentiment classifiers trained with this high-level feature representation clearly outperform state-of-the-art methods on a benchmark composed of reviews of 4 types of Amazon products. Furthermore, this method scales well and allowed us to successfully perform domain adaptation on a larger industrial-strength dataset of 22 domains. <s> BIB015 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> In real-world applications of visual recognition, many factors — such as pose, illumination, or image quality — can cause a significant mismatch between the source domain on which classifiers are trained and the target domain to which those classifiers are applied. As such, the classifiers often perform poorly on the target domain. Domain adaptation techniques aim to correct the mismatch. Existing approaches have concentrated on learning feature representations that are invariant across domains, and they often do not directly exploit low-dimensional structures that are intrinsic to many vision datasets. In this paper, we propose a new kernel-based method that takes advantage of such structures. Our geodesic flow kernel models domain shift by integrating an infinite number of subspaces that characterize changes in geometric and statistical properties from the source to the target domain. Our approach is computationally advantageous, automatically inferring important algorithmic parameters without requiring extensive cross-validation or labeled data from either domain. We also introduce a metric that reliably measures the adaptability between a pair of source and target domains. For a given target domain and several source domains, the metric can be used to automatically select the optimal source domain to adapt and avoid less desirable ones. Empirical studies on standard datasets demonstrate the advantages of our approach over competing methods. <s> BIB016 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> One of the most important challenges in machine learning is performing effective learning when there are limited training data available. However, there is an important case when there are sufficient training data coming from other domains (source). Transfer learning aims at finding ways to transfer knowledge learned from a source domain to a target domain by handling the subtle differences between the source and target. In this paper, we propose a novel framework to solve the aforementioned knowledge transfer problem via low-rank representation constraints. This is achieved by finding an optimal subspace where each datum in the target domain can be linearly represented by the corresponding subspace in the source domain. Extensive experiments on several databases, i.e., Yale B, CMU PIE, UB Kin Face databases validate the effectiveness of the proposed approach and show the superiority to the existing, well-established methods. <s> BIB017 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> Cross-domain learning methods have shown promising results by leveraging labeled patterns from the auxiliary domain to learn a robust classifier for the target domain which has only a limited number of labeled samples. To cope with the considerable change between feature distributions of different domains, we propose a new cross-domain kernel learning framework into which many existing kernel methods can be readily incorporated. Our framework, referred to as Domain Transfer Multiple Kernel Learning (DTMKL), simultaneously learns a kernel function and a robust classifier by minimizing both the structural risk functional and the distribution mismatch between the labeled and unlabeled samples from the auxiliary and target domains. Under the DTMKL framework, we also propose two novel methods by using SVM and prelearned classifiers, respectively. Comprehensive experiments on three domain adaptation data sets (i.e., TRECVID, 20 Newsgroups, and email spam data sets) demonstrate that DTMKL-based methods outperform existing cross-domain learning and multiple kernel learning methods. <s> BIB018 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> In this paper, we introduce a new domain adaptation (DA) algorithm where the source and target domains are represented by subspaces described by eigenvectors. In this context, our method seeks a domain adaptation solution by learning a mapping function which aligns the source subspace with the target one. We show that the solution of the corresponding optimization problem can be obtained in a simple closed form, leading to an extremely fast algorithm. We use a theoretical result to tune the unique hyper parameter corresponding to the size of the subspaces. We run our method on various datasets and show that, despite its intrinsic simplicity, it outperforms state of the art DA methods. <s> BIB019 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> We evaluate whether features extracted from the activation of a deep convolutional network trained in a fully supervised fashion on a large, fixed set of object recognition tasks can be repurposed to novel generic tasks. Our generic tasks may differ significantly from the originally trained tasks and there may be insufficient labeled or unlabeled data to conventionally train or adapt a deep architecture to the new tasks. We investigate and visualize the semantic clustering of deep convolutional features with respect to a variety of such tasks, including scene recognition, domain adaptation, and fine-grained recognition challenges. We compare the efficacy of relying on various network levels to define a fixed feature, and report novel results that significantly outperform the state-of-the-art on several important vision challenges. We are releasing DeCAF, an open-source implementation of these deep convolutional activation features, along with all associated network parameters to enable vision researchers to be able to conduct experimentation with deep representations across a range of visual concept learning paradigms. <s> BIB020 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> Abstract: We present an algorithm that learns representations which explicitly compensate for domain mismatch and which can be efficiently realized as linear classifiers. Specifically, we form a linear transformation that maps features from the target (test) domain to the source (training) domain as part of training the classifier. We optimize both the transformation and classifier parameters jointly, and introduce an efficient cost function based on misclassification loss. Our method combines several features previously unavailable in a single algorithm: multi-class adaptation through representation learning, ability to map across heterogeneous feature spaces, and scalability to large datasets. We present experiments on several image datasets that demonstrate improved accuracy and computational advantages compared to previous approaches. <s> BIB021 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> Cross-domain image synthesis and recognition are typically considered as two distinct tasks in the areas of computer vision and pattern recognition. Therefore, it is not clear whether approaches addressing one task can be easily generalized or extended for solving the other. In this paper, we propose a unified model for coupled dictionary and feature space learning. The proposed learning model not only observes a common feature space for associating cross-domain image data for recognition purposes, the derived feature space is able to jointly update the dictionaries in each image domain for improved representation. This is why our method can be applied to both cross-domain image synthesis and recognition problems. Experiments on a variety of synthesis and recognition tasks such as single image super-resolution, cross-view action recognition, and sketch-to-photo face recognition would verify the effectiveness of our proposed learning model. <s> BIB022 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> In this paper, we tackle the problem of unsupervised domain adaptation for classification. In the unsupervised scenario where no labeled samples from the target domain are provided, a popular approach consists in transforming the data such that the source and target distributions become similar. To compare the two distributions, existing approaches make use of the Maximum Mean Discrepancy (MMD). However, this does not exploit the fact that probability distributions lie on a Riemannian manifold. Here, we propose to make better use of the structure of this manifold and rely on the distance on the manifold to compare the source and target distributions. In this framework, we introduce a sample selection method and a subspace-based method for unsupervised domain adaptation, and show that both these manifold-based techniques outperform the corresponding approaches based on the MMD. Furthermore, we show that our subspace-based approach yields state-of-the-art results on a standard object recognition benchmark. <s> BIB023 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> In this paper, we propose a Generalized Unsupervised Manifold Alignment (GU-MA) method to build the connections between different but correlated datasets without any known correspondences. Based on the assumption that datasets of the same theme usually have similar manifold structures, GUMA is formulated into an explicit integer optimization problem considering the structure matching and preserving criteria, as well as the feature comparability of the corresponding points in the mutual embedding space. The main benefits of this model include: (1) simultaneous discovery and alignment of manifold structures; (2) fully unsuper-vised matching without any pre-specified correspondences; (3) efficient iterative alignment without computations in all permutation cases. Experimental results on dataset matching and real-world applications demonstrate the effectiveness and the practicability of our manifold alignment method. <s> BIB024 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> Much research on human action recognition has been oriented toward the performance gain on lab-collected datasets. Yet real-world videos are more diverse, with more complicated actions and often only a few of them are precisely labeled. Thus, recognizing actions from these videos is a tough mission. The paucity of labeled real-world videos motivates us to "borrow" strength from other resources. Specifically, considering that many lab datasets are available, we propose to harness lab datasets to facilitate the action recognition in real-world videos given that the lab and real-world datasets are related. As their action categories are usually inconsistent, we design a multi-task learning framework to jointly optimize the classifiers for both sides. The general Schatten $$p$$ p -norm is exerted on the two classifiers to explore the shared knowledge between them. In this way, our framework is able to mine the shared knowledge between two datasets even if the two have different action categories, which is a major virtue of our method. The shared knowledge is further used to improve the action recognition in the real-world videos. Extensive experiments are performed on real-world datasets with promising results. <s> BIB025 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> We address the visual categorization problem and present a method that utilizes weakly labeled data from other visual domains as the auxiliary source data for enhancing the original learning system. The proposed method aims to expand the intra-class diversity of original training data through the collaboration with the source data. In order to bring the original target domain data and the auxiliary source domain data into the same feature space, we introduce a weakly-supervised cross-domain dictionary learning method, which learns a reconstructive, discriminative and domain-adaptive dictionary pair and the corresponding classifier parameters without using any prior information. Such a method operates at a high level, and it can be applied to different cross-domain applications. To build up the auxiliary domain data, we manually collect images from Web pages, and select human actions of specific categories from a different dataset. The proposed method is evaluated for human action recognition, image classification and event recognition tasks on the UCF YouTube dataset, the Caltech101/256 datasets and the Kodak dataset, respectively, achieving outstanding results. <s> BIB026 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> Domain adaptation (DA) has gained a lot of success in the recent years in computer vision to deal with situations where the learning process has to transfer knowledge from a source to a target domain. In this paper, we introduce a novel unsupervised DA approach based on both subspace alignment and selection of landmarks similarly distributed between the two domains. Those landmarks are selected so as to reduce the discrepancy between the domains and then are used to non linearly project the data in the same space where an efficient subspace alignment (in closed-form) is performed. We carry out a large experimental comparison in visual domain adaptation showing that our new method outperforms the most recent unsupervised DA approaches. <s> BIB027 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> Recent studies reveal that a deep neural network can learn transferable features which generalize well to novel tasks for domain adaptation. However, as deep features eventually transition from general to specific along the network, the feature transferability drops significantly in higher layers with increasing domain discrepancy. Hence, it is important to formally reduce the dataset bias and enhance the transferability in task-specific layers. In this paper, we propose a new Deep Adaptation Network (DAN) architecture, which generalizes deep convolutional neural network to the domain adaptation scenario. In DAN, hidden representations of all task-specific layers are embedded in a reproducing kernel Hilbert space where the mean embeddings of different domain distributions can be explicitly matched. The domain discrepancy is further reduced using an optimal multikernel selection method for mean embedding matching. DAN can learn transferable features with statistical guarantees, and can scale linearly by unbiased estimate of kernel embedding. Extensive empirical evidence shows that the proposed architecture yields state-of-the-art image classification error rates on standard domain adaptation benchmarks. <s> BIB028 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> Deep feature learning has recently emerged with demonstrated effectiveness in domain adaptation. In this paper, we propose a Deep Nonlinear Feature Coding framework (DNFC) for unsupervised domain adaptation. DNFC builds on the marginalized stacked denoising autoencoder (mSDA) to extract rich deep features. We introduce two new elements to mSDA: domain divergence minimization by Maximum Mean Discrepancy (MMD), and nonlinear coding by kernelization. These two elements are essential for domain adaptation as they ensure the extracted deep features to have a small distribution discrepancy and encode data nonlinearity. The effectiveness of DNFC is verified by extensive experiments on benchmark datasets. Specifically, DNFC attains much higher prediction accuracy than state-of-the-art domain adaptation methods. Compared to its basis mSDA, DNFC is able to achieve remarkable prediction improvement and meanwhile converges much faster with a small number of stacked layers. <s> BIB029 </s> Transfer Learning for Cross-Dataset Recognition: A Survey <s> Transferring Criteria <s> This paper presents a novel unsupervised domain adaptation method for cross-domain visual recognition. We propose a unified framework that reduces the shift between domains both statistically and geometrically, referred to as Joint Geometrical and Statistical Alignment (JGSA). Specifically, we learn two coupled projections that project the source domain and target domain data into low-dimensional subspaces where the geometrical shift and distribution shift are reduced simultaneously. The objective function can be solved efficiently in a closed form. Extensive experiments have verified that the proposed method significantly outperforms several state-of-the-art domain adaptation methods on a synthetic dataset and three different real world cross-domain visual recognition tasks. <s> BIB030
In this section, we summarise the most typical criteria for transferring knowledge between datasets, which have been used explicitly or implicitly among the literature. The basic assumptions and the potential categories of knowledge ("what to transfer") can be transferred using each criterion are analysed and presented. Moreover, several commonly used methods or ideas are illustrated under each criterion. The identified criteria include: statistic criterion, geometric criterion, higher-level representation criterion, correspondence criterion, class-based criterion, self labelling, and hybrid criterion. We denote the source dataset as {X s , Y s }, which are draw from distribution P s (X s , Y s ) and the target dataset as {X t , Y t }, which are draw from distribution P t (X t , Y t ). -Statistic Criterion: aims at reducing the statistical distribution shift between datasets using some mechanisms, such as instance re-weighting, feature transfor-mation, and classifier parameter transfer. The statistic criterion generally assumes sufficient data in each dataset to approximate the respective statistic distributions. This criterion has been used for instance-based transfer, feature representation based transfer, classifier-based transfer, and hybrid transfer (i.e. joint instance and feature representation transfer BIB027 ). The most commonly used methods for comparing and thereby reducing distribution shift are summarised as follows. (1) Kullback-Leibler divergence BIB007 ] (2) Jensen-Shannon divergence BIB013 ] (3) Quadratic divergence BIB011 ] (4) Hellinger distance BIB023 ] (5) Mutual information ] where X represents all the data from source and target domain, Q denotes the domain label (i.e. 0 for source domain, and 1 for target domain),q i is the twodimensional posterior probability vector of assigning x i to either the source or the target, given all other data points from the two domains, andq 0 is the estimated prior distribution of domain (i.e.q 0 = 1/n iq i ). By minimizing the mutual information between the data instance X and its (binary) domain label Q, the domain shift is reduced. (6) H divergence  BIB009 ] where H is a hypothesis class on X , and I(h) is the set for which h ∈ H is the characteristic function; that is x ∈ I(h) ⇔ h(x) = 1. H divergence measures the distance between distributions in a hypothesis class H, which can be approximated by the empirical risk of a classifier that discriminates between instances drawn from P s and instances drawn from P t . (7) Maximum Mean Discrepancy (MMD)  ] where X denotes data from all the datasets, φ represents the kernel function that maps the original data to a reproducing kernel Hilbert space (RKHS). The MMD compares the statistic moments of distributions. If the φ is a characteristic kernel (i.e. Gaussian kernels, or Laplace kernels), MMD compares all the orders of statistic moments, making MMD a metric on distributions. -Geometric Criterion: bridges datasets according to their geometrical properties. This criterion assumes domain shift can be reduced using the relationship of geometric structures between datasets and is generally used for feature representation based transfer. The examples of transferring using geometric criterion are as follows. (1) Subspace alignment BIB019 ] Learn a linear mapping to align the source subspace coordinate system to the target one. where the subspaces A s and A t are pre-learned using PCA on source and target domain respectively. (2) Intermediate subspaces BIB014 BIB016 Identify intermediate subspaces between the source and target, and then learn the information from these subspaces to convey the domain changes. The subspaces are identified with the Grassmann manifold G N,d , and source subspace A s and target subspace A t are points on G N,d . To bridge A s and A t , the points on the geodesic paths between them (which are constant velocity curves on a manifold) are sampled to form the intermediate subspaces. Then both source and target data are projected to the obtained intermediate subspaces (either by sampling along the geodesic BIB014 or all of them BIB016 ) to augment the data for help finding the correlations between domains. (3) Manifold alignment (without correspondence) BIB024 Align the manifolds defined by source and target datasets without the correspondence information. Hence, the two manifolds can be aligned geometrically: where the K s and K t are full adjacency matrix (i.e. ns×nt denotes the correspondence matrix that need to be learnt, A s and A t are linear projections of from two datasets that also need to be learnt, and J(A s , X s ) and J(A t , X t ) are geometry preserving terms to preserve the manifold structures of respective domains. -Higher-level Representations: aims at finding higher-level representations that are representative, compact, or invariant between datasets. This criterion does not assume the existence of labelled data, or the existence of correspondence set. It assumes that there exist the domain invariant higher-level representations between datasets. This criterion is generally used for feature representation based transfer. Note that the higher-level representation criterion is commonly used together with other criteria for better transfer, but it is also used independently without any mechanism to reduce the domain divergence explicitly. (1) Sparse coding BIB002 The dictionary is learnt based on the source data, and then apply the learnt dictionary to the target data to obtain the sparse codes of the target data. Step 1: D = arg min Step 2: Z * t = arg min where D is the learnt dictionary on source data, Z s and Z t are the sparse codes. (2) Low-rank representation BIB017 Find a subspace A where each datum in the target domain can be linearly represented by the corresponding subspace in the source domain. Because Z is constrained to be low rank, each target sample is linearly represented by some subspace from a subspace union in the source domain. Hence, the structure information in the source and target domain is considered. (3) Deep Neural Networks Representations BIB020 Use deep neural networks to learn more transferable features by disentangling explanatory factors of variations underlying data samples, and grouping deep features hierarchically according the their relatedness to invariant factors. (4) Stacked Denoising Auto-encoders (SDAs) BIB015 Train the SDAs BIB012 ] to reconstruct the data from all the domains. It has been shown that SDAs can disentangle hidden factors which explain the variations in the input data that are invariant to different domains BIB015 . The key idea is to map the corrupted inputX to a hidden representation, and then decode the hidden representation back to reconstructed vectors in input space: decoder: whereX is the corrupted version of the data X from all the domains, θ ∈ {W, b} and θ ∈ {W , b } are parameters of the auto-encoder, s(·) is the nonlinear squashing-function, Y is the hidden representation of input data, Z is the reconstructed data which are supposed to be as similar as as possible to the uncorrupted input data X, i.e. min X − Z 2 . Then several layers of denoising auto-encoders can be learnt and stacked to constitute the stacked denoising auto-encoders (SDAs) to build deep architectures. (5) Attribute space representation BIB010 Use the human-specified high-level description (attributes) of target objects instead of training images to detect object in an image.  Hence, the posterior of a target class given an image can be calculated as follows Since the attributes are assigned on a per-class basis instead of a per-image basis, the manual effort to add a new object class is much smaller than labelling a huge amount of training data. -Correspondence Criterion: uses the paired correspondence samples from different domains to construct the relationship between domains. Obviously, a set of corresponding samples (i.e. the same object captured from different view angles, or by different sensors) are required for correspondence criterion. This criterion is commonly used for feature representation-based transfer. The typical methods are as follows. (1) Sparse coding with correspondence  The corresponding samples between domains are force to share the same sparse codes: where D s and D t are the dictionaries, Z is the sparse codes. (2) Manifold alignment (with correspondence)  Given a set of correspondence samples set C between domains, learn mapping matrices A s and A t for source and target set respectively to preserve the correspondence relationships after mapping: where J(A s , X s ) and J(A t , X t ) are the manifold regularization terms which are used to preserve the intrinsic manifold structures of source and target domains. -Class-based Criterion: uses the class label information as a guidance for connecting different datasets. Hence, the labelled samples from each dataset are assumed to be available, no matter sufficient or not. This criterion has been used for feature representation-based transfer BIB003  , classifier-based transfer BIB004 BIB025 , and hybrid knowledge transfer (i.e. joint feature-based and classifier-based transfer BIB021 ). Below is the commonly used methods that use the supervision criterion for transferring the source knowledge to the target dataset. (1) Feature augmentation BIB003 Each feature is augmented into three versions of it: a general version, a sourcespecific version and a target-specific version. The augmented source data will contain only general and source-specific versions. The augmented target data contains general and target-specific versions. The rest of the dimensions are appended with zeros. The augmented features are as follows, where x g is the general version, x s is the source specific version, and x t is the target specific version. (2) Metric learning  Learn a metric such that the distance of samples from different domains with same labels are close while the distance of samples from different domains with different labels are far away: is the pairwise distance for some W , and W is the matrix that needs to be learnt. (3) Linear Discriminative Model BIB004 Use the source classifier parameters to regularize the target classifier parame-ters: where l denotes some loss functions over data and labels, and R denotes some regularization between source and target classifiers. (4) Bayesian Model BIB001 Present the source knowledge as a prior probability in the space of target model parameters. Specifically, the "general knowledge" from source domain categories are extracted and then represented in the form of a prior probability density function in the space of model parameters. Given a small training set in the target domain, this knowledge can be updated and produce a posterior density. -Self Labelling: uses the source domain samples to train an initial model, which is used to obtain the pseudo labels of target domain samples. Then the target samples are incorporated to retrain the model. The procedure is carried iteratively until convergence. The self labelling has been used for classifier-based transfer BIB005 , and hybrid knowledge transfer BIB006 . (1) Self-training BIB006 ] Initialize the target model parameters θ l using the source data (or re-weighted source data), i.e. θ l = arg max θ xs∈Xs log p(x s , y s |θ). The pseudo labels of target samples can be obtained using the initial model, then use EM algorithm to iteratively refine the target model by incorporating some mechanisms (i.e. assign small weight to source samples that are dissimilar to target samples): M step : θ l+1 = arg max where w(x t ) and w(x t ) are assigned weights to each target and source samples using some mechanisms. -Hybrid Criterion: combines two or more above criteria for better transferring of knowledge. The combination of criteria is generally used for feature representation transfer, classifier transfer, and hybrid knowledge transfer. Several examples of combination are (1) Correspondence + Higher-level representation BIB022 (2) Higher-level representation + Statistic BIB028 BIB029 ] (3) Statistic + Geometric BIB030 (4) Statistic + Self labelling BIB005 (5) Correspondence + Class-based BIB008 (6) Statistic + Class-based BIB018 ] (7) Higher-level representation + Class-based BIB026 Based on the summary of different transferring criteria and the categories of "what to transfer", below, we discuss what kinds of methods can be used for boosting current task in the four different scenarios (homogeneous feature spaces and label spaces, heterogeneous label spaces, heterogeneous label spaces, and heterogeneous feature spaces and label spaces) defined in Section 2.1 as well as corresponding problems defined by other dataset properties. An overview of the taxonomy can be found in Figure 1 . In each problem, the related work is reviewed according to transferring criteria: statistic criterion, geometric criterion, higher-level representations criterion, correspondence criterion, class-based criterion, self labelling, and hybrid criterion. Then the methods under each criterion are further subdivided into "what to transfer": instance-based transfer, feature representation-based transfer, classifier-based transfer, and hybrid knowledge transfer.