There is a strong incentive to develop versatile learning techniques that can transfer the knowledge of class-separability from a labeled source domain to an unlabeled target domain in the presence of a domain-shift. Existing domain adaptation (DA) approaches are not equipped for practical DA scenarios as a result of their reliance on the knowledge of source-target label-set relationship (e.g. Closed-set, Open-set or Partial DA). Furthermore, almost all the prior unsupervised DA works require coexistence of source and target samples even during deployment, making them unsuitable for incremental, real-time adaptation. Devoid of such highly impractical assumptions, we propose a novel two-stage learning process. Initially, in the procurement-stage, the objective is to equip the model for future source-free deployment, assuming no prior knowledge of the upcoming category-gap and domain-shift. To achieve this, we enhance the model’s ability to reject out-of-source distribution samples by leveraging the available source data, in a novel generative classifier framework. Subsequently, in the deployment-stage, the objective is to design a unified adaptation algorithm capable of operating across a wide range of category-gaps, with no access to the previously seen source samples. To achieve this, in contrast to the usage of complex adversarial training regimes, we define a simple yet effective source-free adaptation objective by utilizing a novel instance-level weighing mechanism, named as Source Similarity Metric (SSM). A thorough evaluation shows the practical usability of the proposed learning framework with superior DA performance even over state-of-the-art source-dependent approaches. Deep learning models have proven to be highly successful over a wide variety of tasks (Krizhevsky et al., 2012; Ren et al., 2015) . However, a majority of these remain heavily dependent on access to a huge amount of labeled samples to achieve a reliable level of generalization. A recognition model trained on a certain distribution of labeled samples (source domain) often fails to generalize when deployed in a new environment (target domain) in the presence a discrepancy in the input distribution (Shimodaira, 2000) . Domain adaptation (DA) algorithms seek to minimize this discrepancy either by learning a domain invariant feature representation (Long et al., 2015; Kumar et al., 2018; Ganin et al., 2016; Tzeng et al., 2015) , or by learning independent domain transformations (Long et al., 2016) to a common latent representation through adversarial distribution matching (Tzeng et al., 2017; Nath Kundu et al., 2018) , in the absence of target label information. Most of the existing approaches (Zhang et al., 2018c; Tzeng et al., 2017 ) assume a common label-set shared between the source and target domains (i.e. C s = C t ), which is often regarded as Closed-Set DA (see Fig. 1 ). Though this assumption helps to analyze various insights of DA algorithms, such an assumption rarely holds true in real-world scenarios. Recently researchers have independently explored two broad adaptation settings by partly relaxing the above assumption. In the first kind, Partial DA (Zhang et al., 2018b; Cao et al., 2018a; b) , the target label space is considered as a subset of the source label space (i.e. C t ⊂ C s ). This setting is more suited for large-scale universal source datasets, which will almost always subsume the label-set of a wide range of target domains. However, the availability of such a universal source is highly questionable for a wide range of input domains and tasks. In the second kind, regarded as Open-set DA (Baktashmotlagh et al., 2019; Ge et al., 2017) , the target label space is considered as a superset of the source label space (i.e. C t ⊃ C s ). The major challenge in this setting is attributed to detection of target samples from the unobserved categories in a fully-unsupervised scenario. Apart from the above two extremes, certain works define a partly mixed scenario by allowing "private" label-set for both source and target domains (i.e. C s \ C t = ∅ and C t \ C s = ∅) but with extra supervision such as few-shot labeled data (Luo et al., 2017) or access to the knowledge of common categories (Panareda Busto & Gall, 2017) . Most of the prior approaches consider each scenario in isolation and propose independent solutions. Thus, they require access to the knowledge of label-set relationship (or category-gap) to carefully choose a DA algorithm, which would be suitable for the problem in hand. Furthermore, all the prior unsupervised DA works require coexistence of source and target samples even during deployment, hence not source-free. This is highly impractical, as labeled source data may not be accessible after deployment due to several reasons such as, privacy concerns, restricted access to proprietary data, accidental loss of source data or other computational limitations in real-time deployment scenarios. Acknowledging the aforementioned shortcomings, we propose one of the most convenient DA frameworks which is ingeniously equipped to address source-free DA for all kinds of label-set relationships, without any prior knowledge of the associated category-gap (i.e. universal-DA). We not only focus on identifying the key complications associated with the challenging problem setting, but also devise insightful ideas to tackle such complications by adopting learning techniques much different from the available DA literature. This leads us to realize a holistic solution which achieves superior DA performance even over prior source-dependent approaches. a) Comparison with prior arts. We compare our approach with UAN You et al. (2019) , and other prior methods. The results are presented in Table 1 and Table 2 . Clearly, our framework achieves state- Relative freq. We have introduced a novel source-free, universal domain adaptation framework, acknowledging practical domain adaptation scenarios devoid of any assumption on the source-target label-set relationship. In the proposed two-stage framework, learning in the Procurement stage is found to be highly crucial, as it aims to exploit the knowledge of class-separability in the most general form with enhanced robustness to out-of-distribution samples. Besides this, success in the Deployment stage is attributed to the well-designed learning objectives effectively utilizing the source similarity criterion. This work can be served as a pilot study towards learning efficient inheritable models in future.
A novel unsupervised domain adaptation paradigm - performing adaptation without accessing the source data ('source-free') and without any assumption about the source-target category-gap ('universal').
{"src": "scitldr-AIC_train", "paper_id": "B1gd0nEFwS"}