Upload 8 files

README.md

@@ -0,0 +1,49 @@
+---
+license: apache-2.0
+library_name: timm
+---
+# WD ViT Tagger v3
+
+Supports ratings, characters and general tags.
+
+Trained using https://github.com/SmilingWolf/JAX-CV.  
+TPUs used for training kindly provided by the [TRC program](https://sites.research.google/trc/about/).
+
+## Dataset
+Last image id: 7220105  
+Trained on Danbooru images with IDs modulo 0000-0899.  
+Validated on images with IDs modulo 0950-0999.  
+Images with less than 10 general tags were filtered out.  
+Tags with less than 600 images were filtered out.
+
+## Validation results
+`v2.0: P=R: threshold = 0.2614, F1 = 0.4402`  
+`v1.0: P=R: threshold = 0.2547, F1 = 0.4278`
+
+## What's new
+Model v2.0/Dataset v3:  
+Trained for a few more epochs.  
+Used tag frequency-based loss scaling to combat class imbalance.
+
+Model v1.1/Dataset v3:  
+Amended the JAX model config file: add image size.  
+No change to the trained weights.
+
+Model v1.0/Dataset v3:  
+More training images, more and up-to-date tags (up to 2024-02-28).  
+Now `timm` compatible! Load it up and give it a spin using the canonical one-liner!  
+ONNX model is compatible with code developed for the v2 series of models.  
+The batch dimension of the ONNX model is not fixed to 1 anymore. Now you can go crazy with batch inference.  
+Switched to Macro-F1 to measure model performance since it gives me a better gauge of overall training progress.
+
+# Runtime deps
+ONNX model requires `onnxruntime >= 1.17.0`
+
+# Inference code examples
+For timm: https://github.com/neggles/wdv3-timm  
+For ONNX: https://huggingface.co/spaces/SmilingWolf/wd-tagger  
+For JAX: https://github.com/SmilingWolf/wdv3-jax
+
+## Final words
+Subject to change and updates.
+Downstream users are encouraged to use tagged releases rather than relying on the head of the repo.

config.json

@@ -0,0 +1,39 @@
+{
+  "architecture": "vit_base_patch16_224",
+  "num_classes": 10861,
+  "num_features": 768,
+  "global_pool": "avg",
+  "model_args": {
+    "img_size": 448,
+    "class_token": false,
+    "global_pool": "avg",
+    "fc_norm": false,
+    "act_layer": "gelu_tanh"
+  },
+  "pretrained_cfg": {
+    "custom_load": false,
+    "input_size": [
+      3,
+      448,
+      448
+    ],
+    "fixed_input_size": false,
+    "interpolation": "bicubic",
+    "crop_pct": 1.0,
+    "crop_mode": "center",
+    "mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "num_classes": 10861,
+    "pool_size": null,
+    "first_conv": null,
+    "classifier": null
+  }
+}

model.msgpack

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8efa9946a6b967cb0566d5dc2ac6cf736344224273365e8082d1d49b0d987ae9
+size 134

model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7a2a3606a12e94a6a66331df2a3558958332565647f0f20be2e2fd56a37d73b
+size 134

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d38d2378878efe09139d020e80d264861e3cb1e8472448029abf3785763c264d
+size 134

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc9b75e28cbb1f6c342fac49c46bbdff7ba801b3a519ed4e002ddedcc181a0e5
+size 378459814

sw_jax_cv_config.json

@@ -0,0 +1,14 @@
+{
+    "image_size": 448,
+    "model_name": "vit_base",
+    "model_args": {
+        "patch_size": 16,
+        "num_classes": 10861,
+        "num_layers": 12,
+        "embed_dim": 768,
+        "mlp_dim": 3072,
+        "num_heads": 12,
+        "drop_path_rate": 0.1,
+        "layer_norm_eps": 1e-05
+    }
+}