Removing cascaded bounding box refinement.
Sparse RCNN follows the cascaded bounding box regression in Cascade RCNN [1]}, where each decoder layer iteratively refines the bounding box predictions made by the previous layer. We remove it in this stage and as expected, the model performance degrades to some extent.
Although not adopted by most detection transformers like DETR and CondDETR, the cascaded bounding box refinement can be naturally included in their cascaded decoder layers.
