Label Assignment.
Detection transformers view the object detection task as a set prediction problem and perform deep supervision [1]} on predictions made by each decoder layer. Specifically, the labels set can be denoted as \(y=\left\lbrace y_{1}, \ldots , y_{M}, \varnothing , \ldots , \varnothing \right\rbrace \) , where \(M\)  denotes the number of foreground objects in the image and the \(\varnothing \)  (no object) pads the label set to a length of \(N\) . Correspondingly, the output of each decoder layer can be written as \(\hat{y}=\left\lbrace \hat{y}_{i}\right\rbrace _{i=1}^{N}\) . During label assignment, detection transformers search for a permutation \(\tau \in T_{N}\)  with the minimum matching cost:
\(\hat{\tau }=\underset{\tau \in T_{N}}{\arg \min } \sum _{i}^{N} \mathcal {L}_{\text{match}}\left(y_{i}, \hat{y}_{\tau (i)}\right),\) 
