We follow the setting of [1]} and train two metric learning models with Multiple Similarity (MS) loss [1]} and Triplet loss [3]}, respectively. As shown in Table REF , although they have almost the same accuracy on the training set, there is a big gap between their generalization abilities on the validation set. Before checking the result on the validation set, is there any clue in the training set for such gap? Despite the similar training accuracies, the predictions of two models are actually based on different regions from the activation maps generated by our activation decomposition framework (see Fig. REF ). The activation maps also provide valuable insights on the model generalization ability to the validation/test set. It is evident from Fig. REF , the model with MS loss is more focused on the salient object area than the one with Triplet loss, which leads to better validation performance as confirmed in Table REF . The quantitative results of localization in Table REF  also support the fact that “Triplet" model is more likely to focus on the background region rather than the object as we witness the localization accuracy drop for Triplet loss. Although “Decomposition+Bias" can be more sensitive to different loss functions, implying that the bias term does provide valuable information on the overall map, both settings of the proposed method have the same trend on accuracy. Therefore, our activation decomposition framework can shed light on the generalization ability of loss functions for metric learning, thereby providing an useful tool for model diagnosis.
