|
--- |
|
license: other |
|
license_name: apple-sample-code-license |
|
license_link: LICENSE |
|
--- |
|
|
|
A CLIP (Contrastive Language-Image Pre-training) model trained on DFN-2B. |
|
Data Filtering Networks (DFNs) are small networks used to automatically filter large pools of uncurated data. |
|
This model was trained on 2B images that were filtered from a pool of 12.8B uncurated image-text pairs |
|
(12.8B image-text pairs from CommonPool-12.8B). |
|
|
|
This model has been converted to PyTorch from the original JAX checkpoints from Axlearn (https://github.com/apple/axlearn). |
|
These weights are directly usable in OpenCLIP (image + text). |
|
|
|
|
|
## Model Details |
|
|
|
- **Model Type:** Contrastive Image-Text, Zero-Shot Image Classification. |
|
- **Dataset:** DFN-2b |
|
- **Papers:** |
|
- Data Filtering Networks: https://arxiv.org/abs/2309.17425 |
|
- **Examples Seen:** 39B |
|
|
|
|
|
## Model Metrics |
|
| Eval Dataset | Metric | |
|
|:-----------------------|---------:| |
|
| ImageNet 1k | 0.8219 | |
|
| Caltech-101 | 0.9500 | |
|
| CIFAR-10 | 0.9864 | |
|
| CIFAR-100 | 0.8934 | |
|
| CLEVR Counts | 0.3403 | |
|
| CLEVR Distance | 0.2321 | |
|
| Country211 | 0.3198 | |
|
| Describable Textures | 0.6681 | |
|
| EuroSAT | 0.6819 | |
|
| FGVC Aircraft | 0.4829 | |
|
| Food-101 | 0.9498 | |
|
| GTSRB | 0.6329 | |
|
| ImageNet Sketch | 0.7043 | |
|
| ImageNet v2 | 0.7570 | |
|
| ImageNet-A | 0.6745 | |
|
| ImageNet-O | 0.3605 | |
|
| ImageNet-R | 0.9184 | |
|
| KITTI Vehicle Distance | 0.2391 | |
|
| MNIST | 0.8745 | |
|
| ObjectNet | 0.7477 | |
|
| Oxford Flowers-102 | 0.8784 | |
|
| Oxford-IIIT Pet | 0.9611 | |
|
| Pascal VOC 2007 | 0.8472 | |
|
| PatchCamelyon | 0.6418 | |
|
| Rendered SST2 | 0.5815 | |
|
| RESISC45 | 0.7300 | |
|
| Stanford Cars | 0.9465 | |
|
| STL-10 | 0.9889 | |
|
| SUN397 | 0.7594 | |
|
| SVHN | 0.6573 | |
|
| Flickr | 0.8467 | |
|
| MSCOCO | 0.5957 | |
|
| WinoGAViL | 0.5551 | |
|
| iWildCam | 0.1857 | |
|
| Camelyon17 | 0.6540 | |
|
| FMoW | 0.1824 | |
|
| Dollar Street | 0.6822 | |
|
| GeoDE | 0.9253 | |
|
| **Average** | **0.68039** | |
|
|
|
## Citation |
|
```bibtex |
|
@article{fang2023data, |
|
title={Data Filtering Networks}, |
|
author={Fang, Alex and Jose, Albin Madappally and Jain, Amit and Schmidt, Ludwig and Toshev, Alexander and Shankar, Vaishaal}, |
|
journal={arXiv preprint arXiv:2309.17425}, |
|
year={2023} |
|
} |
|
|
|
``` |
|
|
