The Invariant Risk Minimization (IRM) framework is a promising step to address OOD generalization [1]}, which consists of learning a data representation \(\phi \)  that captures invariant features, followed by training a classifier on the invariant features such that it is close to optimal for all environments. The IRM classifier can perform well in OOD tasks if the conditional expectation of the target given the representation features, does not vary across environments [1]}, [3]}, which intuitively means that the data distributions overlap across environments. Due to the additional constraint that the IRM classifier should be optimal for all environments, there is an implicit notion of group fairness in IRM that encourages similar performance across environments, which has been explicitly studied in follow-up works [4]}, [5]}. However, it is unclear how IRM performance is affected if environments do not overlap adequately. Nevertheless, it is essential to overcome any potential shortcomings of IRM in this setting, since minimally overlapping environments often occur in real-world applications. For instance, in discussions in ethnic community forums, while some linguistic features are shared among communities, there are additional features that are specific only to certain intra-community comments but not outside the community [6]}, [7]}. Thus, one would expect the performance in a downstream task such as hate-speech detection would degrade if such intra-community (non-invariant) features are discarded by the IRM classifier.
