For both the single- and multi-product samples, the proposal-level features are extracted via the bottom-up-attention network [1]} pre-trained on the Visual Genome dataset [2]}, the captions are used as input and the entity is to set the same values to EGE-CMP.
The COMT layer produces \(f_{v}\)  and \(f_{t}\)  as the feature embeddings of the visual and textual inputs, respectively, during inference.
Then, \(f_{v}\)  and \(f_{t}\)  are concatenated and normalized to get the instance's joint representations.
Additionally, since the Text/Visual and Text/Entity Transformers are supervised using cross-modal contrastive loss, we could believe that concatenating the feature embeddings of COMTs for retrieval is useful.
Our retrieval model then uses the generated features as input.
After generating the cosine similarity matrix between an instance and the \(gallery\)  set samples, the single-product samples with the greatest similarity are selected and returned for each query for both multi-product and identical-product retrieval tasks.
