According to the collection source, visual-language pre-training datasets can be categorized into at least two main groups:
1) General Sources.:
There are several visual-language datasets [1]}, [2]}, [3]}, [4]}, [5]}, [6]} collected from the social media platforms (e.g. Twitter and Facebook).
Twitter100k [1]} is presented for cross-media retrieval with a low degree of supervision.
It is a collection of 100k image-text pairings crawled from the Twitter website.
There are no limitations on the categories of images, and users supply the textual description in a colloquial language.
Similarly, this dataset emphasizes image-level retrieval and employs single-modal input as the query, which is incompatible with instance-level multi-modal retrieval.
INRIA-Websearch [6]} is a cross-modal retrieval dataset that contains 71,478 image-text pairs connected with 353 unique search queries, such as actors, films, and so on.
The images are gathered via internet searches, and the textual description is derived from the text that surrounds the images on websites.
In this dataset, cross-modal retrieval methods are used to tackle the text-to-image search issue.
2) Product Sources. For the E-commerce datasets, RPC checkout [9]} and Dress Retrieval [10]} are crawled from the shopping website as two widely used typical datasets.
RPC [9]} comprises 53,739 photos of individual products and 30,000 photographs of checkout.
Three independent levels of annotation are available: category information, point-level annotations, and bounding boxes.
There is, however, a gap between RPC and real-world settings, since RPC was gathered in a controlled environment and this dataset is devoid of textual information.
Dress Retrieval proposed in [10]} collects obtains noisy labeled data scraped from catalogs of E-commerce websites.
