Causal Balancing for Domain Generalization
While machine learning models rapidly advance the state-of-the-art on various
real-world tasks, out-of-domain (OOD) generalization remains a challenging
problem given the vulnerability of these models to spurious correlations. While
current domain generalization methods usually focus on enforcing certain
invariance properties across different domains by new loss function designs, we
propose a balanced mini-batch sampling strategy to reduce the domain-specific
spurious correlations in the observed training distributions. More
specifically, we propose a two-phased method that 1) identifies the source of
spurious correlations, and 2) builds balanced mini-batches free from spurious
correlations by matching on the identified source. We provide an
identifiability guarantee of the source of spuriousness and show that our
proposed approach provably samples from a balanced, spurious-free distribution
over all training environments. Experiments are conducted on three computer
vision datasets with documented spurious correlations, demonstrating
empirically that our balanced mini-batch sampling strategy improves the
performance of four different established domain generalization model baselines
compared to the random mini-batch sampling strategy.