As discussed before, for datasets CUB and OAI, we set the concept weights \(\gamma \)  to 0.001 and 1 respectively (as reported in [1]}), while for ConceptMNIST we set \(\gamma \)  to 0.5 for all subsequent experiments. The learning rates \(\omega \)  for C-MNIST is 0.01 and \(\omega ^{^{\prime }}\)  is set as 0.005. For CUB, we follow [2]} with \(\omega \)  at 0.01 and \(\omega ^{^{\prime }}\)  0.005 while for OAI \(\omega \)  and \(\omega ^{^{\prime }}\)  is set as 0.0008 and 0.0005 respectively. We use SGD momentum as the optimizer and follow tuning rules in [3]} to set the hyperparameters of the optimizer.
<TABLE>