Removing RoIAlign.
Sparse RCNN, like other detectors in the RCNNs family, samples features from local regions of interest, and then makes predictions based on the sampled sparse features [1]}. By contrast, each content query in DETR aggregates object-specific information directly from the global features map. In this step, we remove the RoIAlign [2]} operation in Sparse RCNN. Note the box target transformation [3]}
is also removed. It can be seen that significant degradation of the model performance occurs, especially under the 50E schedule, the model performance decreases by 8.4 AP. We conjecture that learning to focus on local object regions from the entire feature map is non-trivial. The model requires more data and training epochs to capture the locality properties.
