Update README.md
Browse files
README.md
CHANGED
@@ -3,6 +3,77 @@ tags:
|
|
3 |
- image-classification
|
4 |
- timm
|
5 |
library_name: timm
|
6 |
-
license:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
7 |
---
|
|
|
8 |
# Model card for vit_small_patch8_224.lunit_dino
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
- image-classification
|
4 |
- timm
|
5 |
library_name: timm
|
6 |
+
license: other
|
7 |
+
license_name: lunit-non-commercial
|
8 |
+
license_link: https://github.com/lunit-io/benchmark-ssl-pathology/blob/main/LICENSE
|
9 |
+
datasets:
|
10 |
+
- 1aurent/BACH
|
11 |
+
- 1aurent/NCT-CRC-HE
|
12 |
+
- 1aurent/PatchCamelyon
|
13 |
+
pipeline_tag: image-classification
|
14 |
---
|
15 |
+
|
16 |
# Model card for vit_small_patch8_224.lunit_dino
|
17 |
+
|
18 |
+
A Vision Transformer (ViT) image classification model. \
|
19 |
+
Trained on 33M histology patches from various pathology datasets.
|
20 |
+
|
21 |
+
![](https://github.com/lunit-io/benchmark-ssl-pathology/raw/main/assets/ssl_teaser.png)
|
22 |
+
|
23 |
+
## Model Details
|
24 |
+
|
25 |
+
- **Model Type:** Feature backbone
|
26 |
+
- **SSL Method:** DINO
|
27 |
+
- **Model Stats:**
|
28 |
+
- Params (M): 21.7
|
29 |
+
- Image sizes: 224 × 224 x 3
|
30 |
+
- **Papers:**
|
31 |
+
- Benchmarking Self-Supervised Learning on Diverse Pathology Datasets: https://arxiv.org/abs/2212.04690
|
32 |
+
- **Datasets:**
|
33 |
+
- BACH
|
34 |
+
- CRC
|
35 |
+
- MHIST
|
36 |
+
- PatchCamelyon
|
37 |
+
- CoNSeP
|
38 |
+
- **Original:** https://github.com/lunit-io/benchmark-ssl-pathology
|
39 |
+
- **License:** [lunit-non-commercial](https://github.com/lunit-io/benchmark-ssl-pathology/blob/main/LICENSE)
|
40 |
+
|
41 |
+
## Model Usage
|
42 |
+
|
43 |
+
### Image Embeddings
|
44 |
+
```python
|
45 |
+
from urllib.request import urlopen
|
46 |
+
from PIL import Image
|
47 |
+
import timm
|
48 |
+
|
49 |
+
# get example histology image
|
50 |
+
img = Image.open(
|
51 |
+
urlopen(
|
52 |
+
"https://github.com/owkin/HistoSSLscaling/raw/main/assets/example.tif"
|
53 |
+
)
|
54 |
+
)
|
55 |
+
|
56 |
+
# load model from the hub
|
57 |
+
model = timm.create_model(
|
58 |
+
model_name="hf-hub:1aurent/vit_small_patch8_224.lunit_dino",
|
59 |
+
pretrained=True,
|
60 |
+
).eval()
|
61 |
+
|
62 |
+
# get model specific transforms (normalization, resize)
|
63 |
+
data_config = timm.data.resolve_model_data_config(model)
|
64 |
+
transforms = timm.data.create_transform(**data_config, is_training=False)
|
65 |
+
|
66 |
+
output = model(transforms(img).unsqueeze(0)) # output is (batch_size, num_features) shaped tensor
|
67 |
+
```
|
68 |
+
|
69 |
+
## Citation
|
70 |
+
```bibtex
|
71 |
+
@inproceedings{kang2022benchmarking,
|
72 |
+
author = {Kang, Mingu and Song, Heon and Park, Seonwook and Yoo, Donggeun and Pereira, Sérgio},
|
73 |
+
title = {Benchmarking Self-Supervised Learning on Diverse Pathology Datasets},
|
74 |
+
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
|
75 |
+
month = {June},
|
76 |
+
year = {2023},
|
77 |
+
pages = {3344-3354}
|
78 |
+
}
|
79 |
+
```
|