In this paper, we propose Multi-Scale TRasnformer (MSTR), a transformer-based HOI detector that can exploit multi-scale feature maps for HOI detection.
Inspired by previously proposed deformable attention for standard object detection [1]}, we aim to efficiently explore multi-scale feature maps by attending to only a small number of sampling points generated from the query element instead of calculating the attention values for the entire spatial dimension.
Yet, we found out in our preliminary experiments that directly applying naïve deformable attention in HOI detection leads to a serious performance drop.
