Next, We first investigate a realistic problem: how to perform instance-level Instance-level product retrieval refers to the retrieval of all single products that existed in a product portfolio image. fine-grained product retrieval given the weakly annotated multi-modality data on a vast scale?
In Figure REF , we give a comparison of different retrieval patterns to better show the challenge.
From the figure, we could find that image-level retrieval tends to match retrieval objects from all aspects since it is not capable of discriminating between distinct instances, but multi-modal instance-level retrieval is much more capable of searching for different types of commodities among multi multimodality data.
Although the challenge has generality and practical importance, it is under-examined enough owing to a lack of high-quality datasets fitting the real scenarios and a clear description of the issue.
According to relevant studies of product retrieval, uni-modal [1]}, [2]}, [3]}, [4]} and cross-modal retrieval [5]}, [6]}, [7]}, [8]}, [9]}, [10]} take as input unimodal information, e.g., a visual feature or a textual description, and performs a matching search between individual samples.
Unfortunately, such retrieval strategies severely limit their applicability in many scenarios where both the queries and the targets include multi-modal information.
More importantly, past researches concentrate on comparatively straightforward scenario, i.e., image-level Image-level product retrieval refers to recognizing a specific product instance in a single-product image. retrieval for single-product images [11]}, [12]}, leaving the instance-level aspect of retrieval untouched.
<FIGURE>