Introducing transformer encoder.
In DETR, the transformer encoder can be regarded as the neck in the detector, which is used to enhance the features extracted by the backbone. After removing the FPN neck, we add the transformer encoder neck to the model. Note that the projection on the backbone features and sine position encoding are also introduced at the same time, as did in DETR. It can be seen that the AP result decreases at 50E schedule while improves at 300E schedule. We conjecture that similar to ViT [1]}, the attention mechanism in the encoder requires longer training epochs to converge and manifest its advantages, due to quadratic complexity and lack of inductive biases.
