arxiv:1503.01817 region:us
The YFCC100M is one of the largest publicly and freely useable multimedia collection, containing  the metadata of around 99.2 million photos and 0.8 million videos from Flickr, all of which were shared under one of the various Creative Commons licenses.

This version is a subset defined in openai/CLIP.
@article{thomee2016yfcc100m,
author = "Bart Thomee and David A. Shamma and Gerald Friedland and Benjamin Elizalde and Karl Ni and Douglas Poland and Damian Borth and Li-Jia Li",
title = "{YFCC100M}: The New Data in Multimedia Research",
journal = "Communications of the {ACM}",
volume = "59",
number = "2",
pages = "64--73",
year = "2016",
url = "http://cacm.acm.org/magazines/2016/2/197425-yfcc100m/fulltext",
}
# YFCC100M subset from OpenAI

Subset of [YFCC100M](https://arxiv.org/abs/1503.01817) used by OpenAI for [CLIP](https://github.com/openai/CLIP/blob/main/data/yfcc100m.md), filtered to contain only the images that we could retrieve.

| Split | train | validation |
| --- | --- | --- |
| Number of samples | 14,808,859 | 16,374 |
| Size | 1.9 TB | 2.1 GB |

Features:
* from the original dataset: `title`, `description`, `photoid`, `uid`, `unickname`, `datetaken`, `dateuploaded`, `capturedevice`, `usertags`, `machinetags`, `longitude`, `latitude`, `accuracy`, `pageurl`, `downloadurl`, `licensename`, `licenseurl`, `serverid`, `farmid`, `secret`, `secretoriginal`, `ext`, `marker`, `key`
* `img`: image content, can be loaded with `PIL.Image.open(io.BytesIO(item['img']))`
* `title_clean` and `description_clean`: derived from `title` and `description` using `clean_text` function detailed below

```python
def clean_text(text):
    # decode url
    text = urllib.parse.unquote_plus(text)
    # remove html tags
    text = re.sub('<[^<]+?>', '', text)
    # remove multiple spaces + "\r" + "\n" + "\t"
    text = " ".join(text.split())
    return text
```