Also, to further validate the robustness of loss functions, we plan to experiment models on more noisy data. The reason for max-margin's failure on Flikr30k is more likely that the training set is too small - so the model easily overfits. However, the dataset (Flikr30k) itself is rather clean and accurate. It makes more sense to experiment with a noisy dataset with weak text-image correspondence or even false labels. We have two types of candidates for this need: 1) academic datasets that contain “foil” [1]} or adversarial samples [2]}; 2) a real-world text-image dataset such as a news article-image dataset [3]}, [4]}.
