Ryukijano commited on
Commit
c80023e
1 Parent(s): aa67099

Update README.md

Browse files

Added description from the model card.

Files changed (1) hide show
  1. README.md +19 -0
README.md CHANGED
@@ -15,6 +15,25 @@ tags:
15
  - Diffusers
16
  - Controlnet
17
  - Stable-Diffusion
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
  ---
19
 
20
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
15
  - Diffusers
16
  - Controlnet
17
  - Stable-Diffusion
18
+ datasets:
19
+ - animelover/danbooru2022
20
+ ---
21
+
22
+ Experimental proof of concept made for the [Huggingface JAX/Diffusers community sprint](https://github.com/huggingface/community-events/tree/main/jax-controlnet-sprint)
23
+
24
+ [Demo available here](https://huggingface.co/spaces/Ryukijano/CatCon-One-Shot-Controlnet-SD-1-5-b2)
25
+ [My teammate's demo is available here] (https://huggingface.co/spaces/Cognomen/CatCon-Controlnet-WD-1-5-b2)
26
+
27
+ This is a controlnet for the Stable Diffusion checkpoint [Waifu Diffusion 1.5 beta 2](https://huggingface.co/waifu-diffusion/wd-1-5-beta2) which aims to guide image generation by conditioning outputs with patches of images from a common category of the training target examples. The current checkpoint has been trained for approx. 100k steps on a filtered subset of [Danbooru 2021](https://gwern.net/danbooru2021) using artists as the conditioned category with the aim of learning robust style transfer from an image example.
28
+
29
+ Major limitations:
30
+
31
+ - The current checkpoint was trained on 768x768 crops without aspect ratio checkpointing. Loss in coherence for non-square aspect ratios can be expected.
32
+ - The training dataset is extremely noisy and used without filtering stylistic outliers from within each category, so performance may be less than ideal. A more diverse dataset with a larger variety of styles and categories would likely have better performance.
33
+ - The Waifu Diffusion base model is a hybrid anime/photography model, and can unpredictably jump between those modalities.
34
+ - As styling is sensitive to divergences in model checkpoints, the capabilities of this controlnet are not expected to predictably apply to other SD 2.X checkpoints.
35
+
36
+ Waifu Diffusion 1.5 beta 2 is licensed under [Fair AI Public License 1.0-SD](https://freedevproject.org/faipl-1.0-sd/). This controlnet imposes no restrictions beyond the MIT license, but it cannot be used independently of a base model.
37
  ---
38
 
39
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference