The collected pixel-level pseudo-labels are used to train a U-Net style localization network [1]}, as illustrated in Fig.REF . It takes the image as input, and yields a foreground and a background CAMs for localization. To train this model, we consider a composite loss that leverages local and global constraints. Local constraints aim to guide learning at pixel level. To this end, standard partial cross-entropy is used to exploit sampled pixel-wise pseudo-labels. To produce a consistent CAM that is well aligned with object boundaries, we also include a Conditional Random Field (CRF) loss [2]}. To further ensure that the localized object is well aligned with the image-class label, we add a global constraint over the foreground CAM. In particular, we feed the product of the image by the foreground CAM to the pretrained classifier. Then, the classifier response is maximized with respect to the true image-class. During training, only the weights of the localization network are optimized. The SST and classifier models are frozen. At the end of the training, only the localization network and classifier models are retained for classification and localization tasks. The rest of this section provides more details on DiPS components.
