The feed-forward block is a two-layers multi-layer perceptron (MLP) with the GELU [1]} non-linearity function. Following the forward pass through \(L\)  such encoders, the final output of the transformer module is a set of attentive region features \(R^{att}_{feat} = \lbrace f^{att}_{i}\rbrace _{i\in N_{roi^+}}\) . By virtue of the self-attention mechanism, we learn the cross-correlation between positive region features, and in turn the ROIs that contribute salient information for prototype computation.
