We used the triplet loss with hard example mining in the mini-batch [1]}. We also investigated various ranking losses such as SoftTriple loss [2]}, Multi-Similarity loss [3]}, and Circle loss [4]}, but found no gain in performance. Note that we did not optimize the hyperparameters for each loss which means additional parameter optimization might provide better performance for the task. For the image transformation, the image was resized to 224 x 224 and padded maintaining the ratio followed by a random horizontal flip and a random affine including rotation, translation, and scaling. We observed no improvements by adding random erasing. We used the AdamW optimizer [5]} with the default setting provided by PyTorch if not stated otherwise. We set the hyper-parameters \(lr\)  = 0.00011148, \(beta\)  = 0.47, and \(batch size\)  = 32, where \(lr\)  is the initial learning rate, \(beta\)  is a coefficient for AdamW. The learning rate is decayed with a factor of 0.474 every 10 epochs during the 80 epochs of training in total. We applied small \(lr\)  for the image encoder by lowering the \(lr\)  with a factor of 0.48. The image and the text encoder were not pretrained on any external datasets. To describe the preprocessing the texts, we add special token "[CLS]" at the beginning and join the sentences with special token "[SEP]". If the tokenized word is not found in our vocabulary, the word is set to the token "[UNK]". We randomly shuffle the order of the sentences every epoch for better generalization.
<TABLE><FIGURE>