This work was performed under the following financial assistance award: 60NANB18D279 from U.S. Department of Commerce, National Institute of Standards and Technology, and also funding from SAP SE
[width=.17 ] img_orig_000038.jpg & [width=.17 ] img_reg_patch_000038_orig_014_target_017_pred_017_prob_100.png & [width=.17 ] mask_reg_patch_000038_orig_014_target_017_pred_017.JPEG & [width=.17 ] img_our_patch_000038_orig_014_target_017__pred_017_prob_86.png & [width=.17 ] mask_our_patch_000038_orig_014_target_017_pred_017.png Person & Sofa & Sofa & Sofa & Sofa [width=.17 ] img_orig_000145.jpg & [width=.17 ] img_reg_patch_000145_orig_018_target_008_pred_008_prob_100.png & [width=.17 ] mask_reg_patch_000145_orig_018_target_008_pred_008.JPEG & [width=.17 ] img_our_patch_000145_orig_018_target_008__pred_008_prob_97.png & [width=.17 ] mask_our_patch_000145_orig_018_target_008_pred_008.png Train & Chair & Chair & Chair & Chair [width=.17 ] img_orig_000168.jpg & [width=.17 ] img_reg_patch_000168_orig_012_target_003_pred_003_prob_100.png & [width=.17 ] mask_reg_patch_000168_orig_012_target_003_pred_003.JPEG & [width=.17 ] img_our_patch_000168_orig_012_target_003__pred_003_prob_91.png & [width=.17 ] mask_our_patch_000168_orig_012_target_003_pred_003.png Horse & Boat & Boat & Boat & Boat [width=.17 ] img_orig_002007.jpg & [width=.17 ] img_reg_patch_002007_orig_000_target_017_pred_017_prob_100.png & [width=.17 ] mask_reg_patch_002007_orig_000_target_017_pred_017.JPEG & [width=.17 ] img_our_patch_002007_orig_000_target_017__pred_017_prob_99.png & [width=.17 ] mask_our_patch_002007_orig_000_target_017_pred_017.png Aeroplane & Sofa & Sofa & Sofa & Sofa @cite_6 network for VOC dataset. The predicted label is written under each image, the attack was successful for all images, and Grad-CAM is always computed for the target category. GAIN @math is particularly designed to produce better Grad-CAM visualizations using direct supervision of the Grad-CAM output.
@cite_6
2788250925
Weakly supervised learning with only coarse labels can obtain visual explanations of deep neural network such as attention maps by back-propagating gradients. These attention maps are then available as priors for tasks such as object localization and semantic segmentation. In one common framework we address three shortcomings of previous approaches in modeling such attention maps: We (1) first time make attention maps an explicit and natural component of the end-to-end training, (2) provide self-guidance directly on these maps by exploring supervision form the network itself to improve them, and (3) seamlessly bridge the gap between using weak and extra supervision if available. Despite its simplicity, experiments on the semantic segmentation task demonstrate the effectiveness of our methods. We clearly surpass the state-of-the-art on Pascal VOC 2012 val. and test set. Besides, the proposed framework provides a way not only explaining the focus of the learner but also feeding back with direct guidance towards specific tasks. Under mild assumptions our method can also be understood as a plug-in to existing weakly supervised learners to improve their generalization performance.