However, as mentioned above, there is a significant limitation in previous feature-space adversarial learning methods [1]}, [2]}, [3]}: there is no explicit constraint to prevent the network from encoding task-independent nuisance factors into the latent features, which makes the adaptation difficult and the adversarial training unstable. To handle the issue, we propose to distill the task-dependent parts from the crude features and conduct the adaptation based on these “purified" representations, thus helping the feature adaptation and stabilizing the adversarial training.
