We show the architecture of our attention model in Fig. REF (e), which is inspired by the network in [1]} and deeply-supervised network in [2]}. The network incorporates multi-scale and multi-level attention information from different layers, which is learned in a deeply supervised manner. The major difference between the adopted network and previous models is that, our network provides integrated direct supervision to the hidden layers, rather than the standard approach of providing supervision only at the output layer and propagating this supervision back to earlier layers. The multi-level and multi-scale saliency is explicitly learned from different layers with corresponding supervision. Such hidden layer supervision brings improvement in both performance and robustness of features, as discussed in [2]}, [1]}.
