Although some studies have shown that deep networks can learn more transferable representations that disentangle the exploratory factors of variations underlying the data samples and group features hierarchically by their relatedness to invariant factors, Donahue et al. [1]} showed that a domain shift still affects their performance. The deep features would eventually transition from general to specific, and the transferability of the representation sharply decreases in higher layers. Therefore, recent work has addressed this problem by deep da, which combines dl and da. Although shallow methods where deep networks are employed only to extract features and do not help transfer knowledge directly can be considered deep da (such as [2]}), here we focus on a more strict definition. We discuss deep da techniques where the da is in some way embedded in the learning process (i.e., back-propagated) and where the goal is to produce deep domain invariant feature representations. Deep da techniques generally fall into three categories, namely discrepancy-based, adversarial-based and reconstruction-based, that we analyze in the following. It is worth noting that most of these techniques are proposed in the context of the classification task. At the end of this section, we discuss different application examples using deep da, a field which, however, remains more unexplored.
