Our ResNet-50 baseline in row 2 of Table REF , is significantly better than what is usually reported in the literature (row 1). Our attention variants achieve at least 0.7 mAP gains on bounding box detection and at least 0.6 mAP gains on instance segmentation on top of our stronger baselines (denoted by ++ in rows 3, 4 and 6 in Table REF ).
The gain from local attention with block size \(b=8\)  closes half of the mAP gap between the R50 and R101 baselines in detection and 70% of the gap in instance segmentation despite being less than a third of the gap in terms of wall-clock time.
Local attention with \(b=8\)  and \(h=3\)  also improves on top of the deep R101 backbone.
Interestingly, localization of large objects (\(AP^*_l\) ) shows the largest improvement when attention is used.
Larger block sizes (\(b=32\)  in row 4) achieve very close performance to \(b=8\)  while being slower. However, we see that \(b=32\)  does much better than \(b=8\)  on small objects (\(AP^{*}_s\) ).
Future work can combine the best of these two settings. Note that with \(b=32\) , the last two attention layers do global attention since the image is downsampled to \(\frac{1024}{32} = 32\)  pixels in each spatial dimension. Concurrent work, BoTNet [1]}, uses global self-attention in ResNet-Attention hybrids for structured prediction tasks and classification. See [1]} for additional details on the efficacy of global attention for localization tasks
