As GroupDRO  requires explicit domain labels \(g_i\)  for each sample to maximize the worst-case performance, appropriate domain labels are critical for its success. We argue that this is the case for all MDG methods that explicitly leverage domain labels in the dataset  [1]}, [2]}, [3]}, [4]}, [5]}.
We thus make a first attempt at studying the important link between choosing the appropriate group labels and generalization performance.
To this end, we ignore the domain-labels that come with the dataset (e.g., photos, sketch etc. in PACS [6]} dataset) and attempt to re-categorize the data into custom domain groups during training. Our proposed solution, GroupDRO++, alternatively performs unsupervised clustering to re-group samples and minimizes the objective in Eq (REF ). We also introduce a novel regularization strategy to make the latent space amenable for meaningful clustering.
