Removing NMS.
The one-to-many matching between ground-truths and predictions in our label augmentation can provide richer training supervision to alleviate the data-hungry issue. However, it also brings the need for the duplicate removal process. Though the NMS has been proved effective, we hope to remove this post-processing step to maintain the end-to-end property of detection transformers. To achieve this, we provide three possible solutions as follows. Firstly, a lightweight duplicate removal network can be trained along with the model, as did in Relation Network [1]}. Secondly, an additional rank loss [2]} can be applied to regularize the score of the predictions, so that the predictions matched to the original ground-truths can rank higher. In this way, the duplicate removal process is no longer needed. Thirdly, we can apply the label augmentation only on the shallower decoder layers while keeping the supervision on the deep decoder layers unchanged. In this way, the queries at the shallower decoder layers can receive a rich supervision signal, while the deeper decoder layers only select the most promising query for each target instance.
