A survey on heterogeneous transfer learning <s> CL-SCL <s> Discriminative learning methods are widely used in natural language processing. These methods work best when their training and test data are drawn from the same distribution. For many NLP tasks, however, we are confronted with new domains in which labeled data is scarce or non-existent. In such cases, we seek to adapt existing models from a resource-rich source domain to a resource-poor target domain. We introduce structural correspondence learning to automatically induce correspondences among features from different domains. We test our technique on part of speech tagging and show performance gains for varying amounts of source and target training data, as well as improvements in target domain parsing accuracy using our improved tagger. <s> BIB001 </s> A survey on heterogeneous transfer learning <s> CL-SCL <s> We present a new approach to cross-language text classification that builds on structural correspondence learning, a recently proposed theory for domain adaptation. The approach uses unlabeled documents, along with a simple word translation oracle, in order to induce task-specific, cross-lingual word correspondences. We report on analyses that reveal quantitative insights about the use of unlabeled data and the complexity of inter-language correspondence modeling. ::: ::: We conduct experiments in the field of cross-language sentiment classification, employing English as source language, and German, French, and Japanese as target languages. The results are convincing; they demonstrate both the robustness and the competitiveness of the presented ideas. <s> BIB002
Prettenhofer and Stein BIB002 proposed a new HTL method called Cross-Language Structural Correspondence Learning (CL-SCL) for cross-language text classification that is built upon structural correspondence learning (SCL) BIB001 . These transfer learning tasks are inherently heterogeneous in nature as the feature space of a source document is written in a different language as the target. They also have non-overlapping feature spaces due to each language having a distinct word set in their vocabulary. In this case, the goal of such task is to enhance the performance of a target text classification task which contains only unlabeled instances utilizing knowledge from a source domain with amble labeled and unlabeled documents. As mentioned previously, these documents do not share any common features/words as they are from different languages; however, one can link the two of them through the words' semantic meaning. Using this concept as well as SCL theory, this method proposes using word pairs {w s , w t } as pivots which capture correspondences between the source and target languages as they are semantically equivalent. From these correspondences, we can learn a symmetric transformation mapping to discover a common latent feature space which reduces the task into a standard classification problem. This CL-SCL method is comprised of three steps. The first step selects a small set of word pairs to be used as pivots by querying a translation oracle, such as Google Translate, with a source word to finds its corresponding translation in the target vocabulary. With this relation, w s captures the correspondence with the source vocabulary and w t with the target. Both words from these pivots must have good predictive value and occur frequently to be effective. The second step removes these pivot words from a copy of each feature space where then a linear classifier is trained for each pivot using the data from both domains. These pivot classifiers are applied to this copy as to predict whether a pivot word occurs in a document based on the other words in it. This allows us to model the relationship of a pivot word to other words in documents associated to each class. From these, we then compute correlations across pivots with singular value decomposition in step three as to then finally learn a transformation mapping to apply to each input space to discover the latent feature subspace. This method uses less resources than current approaches as it does not require extensive multi-lingual dictionaries or parallel corpus. Experiments performed for cross-language sentiment classification showed the proposed CL-SCL had competitive results.