Recent work [1]} also explores visualization techniques on metric learning, but it only provides qualitative results on several specific architectures. Another variant [2]} replaces the classification score with triplet loss in Grad-CAM framework and deploys weights transfer from the training data in the inference phase. The method requires three samples to form the triplet and its usage in the inference phase is dependent upon the specifically designed transfer procedure, thus is not a general method given only the pretrained model. Moreover, all the aforementioned methods only focus on the overall activation map which is not satisfactory for understanding the model as discussed in Section . Apart from white-box explanation, black-box optimization method, such as perturbation[3]}, needs reformulation for metric learning and can be more computationally expensive if the point-to-point activation map is desired.
