Some recent work have been proposed to extract discriminative localization from class tokens. In particular, TS-CAM [1]} has been proposed to leverage localization in transformers, where the average of all class tokens is multiplied with a semantic-aware map to produce the final activation map.
Since TS-CAM averages an abundance of class-agnostic attention maps, it is prone to localization error by including non-discriminative regions. Additionally, maps from earlier layers attend to background regions, as shown in Fig.REF (a). Therefore, accumulating all maps introduces noise in the final activation map that can reduce the model's localization accuracy. In addition, different transformer-based methods have been proposed[2]}, [3]}, [4]}, [5]}. [2]} propose a spatial calibration module to capture semantic relationships and spatial similarities. Similarly, ViTOL [4]} proposes to incorporate a patch-based attention dropout layer into the transformer attention blocks to improve localization maps. [3]} proposes a relational patch-attention module to enhance local perception quality, and to retain global information. [5]} introduces a mechanism to suppress background regions and focus on the target object. All these methods are unable to minimize loss over a generated activation map, limiting their ability to fully cover an object, and force us to search for an optimal threshold.
