In fact, it is challenging to achieve unbiasedness across the two completely different test distributions due to the domain selection issue [1]}: the two distributions would never co-exist in training — once one is observed, the other is gone — let alone adjusting a fair prior for both of them. In this paper, we propose a novel training paradigm called Cross-Domain Empirical Risk Minimization (xERM) to tackle this challenge. xERM consists of two ERM terms, which are the cross-entropy losses with different supervisions: one is the ground-truth label from the seen imbalanced domain, and the other is the prediction of a balanced model.
The imbalanced domain empirical risk encourages the model to learn from the ground-truth annotations from the imbalanced distribution, which favors the head classes. The balanced domain empirical risk encourages the model to learn from the prediction of a balanced model, which imagines a balanced distribution and prefers the tail classes. These two risks are weighted to take advantage of both, i.e., protect the model training from being neither too “head biased” nor too “tail biased”, and can achieve the unbiasedness (cf. Figure REF ).
Furthermore, we propose a causal theory to ground xERM, which removes the confounding bias caused by domain selection (cf. Section ).
