Spaces:
Running
Definitely not working for new models which is a big deal.
When converting new 2.1 models, the controlnet model fails to run at all while the standard finetuned model converts but when I run it, it throws this error:
The config attributes {'class_embeddings_concat': False, 'cross_attention_norm': None, 'encoder_hid_dim': None, 'mid_block_only_cross_attention': None, 'resnet_out_scale_factor': 1.0, 'resnet_skip_time_act': False, 'time_embedding_act_fn': None} were passed to UNet2DConditionModel, but are not expected and will be ignored. Please verify your config.json configuration file.
It also then returns totally blown out images
Any help would be extremely appreciated because this seems to be the only way to convert these models for use with diffusers.
The only hint I can see in the logs is:
"global_step key not found in model" not sure if related
Another hint is that the unet/config.json for this 2.X model now has:"use_linear_projection": true
while a working 2.X model I trained has it set to false but if I try and repeat that now I get:RuntimeError: weight should have at least three dimensions
Hey @alxdfy,
I just updated the whole space and tested it on the following repos:
- https://huggingface.co/patrickvonplaten/v1-5
- https://huggingface.co/patrickvonplaten/v2-1
- https://huggingface.co/patrickvonplaten/v2-1-base
- https://huggingface.co/patrickvonplaten/v2-0
I don't think it works for controlnet models yet, but we could easily leverage: https://github.com/huggingface/diffusers/blob/eade4308dabc7f7ba75eab508d386b66b3764513/src/diffusers/pipelines/stable_diffusion/convert_from_ckpt.py#L1331
Would you like to open a PR? :-)
Hey @patrickvonplaten really appreciate the help, I've been pulling my hair out the past week trying to figure this out.
Trying to convert this model (sun) https://huggingface.co/ItsPerpetual/perpetual-diffusion/tree/main to diffusers
You can see my efforts converting in this repo (https://huggingface.co/alxdfy/perpetualdiffusionsun) but when I try using it with the controlnet pipeline it's totally blown out while other models work great.
My current working notebook is here https://colab.research.google.com/drive/1MJjU5thLGkcclobd8kT2fUPet_KJ9M3B?usp=sharing
Once again thank you :)
Cool - seems to work now: https://huggingface.co/alxdfy/perpetualdiffusionsun?text=Generate+something (I've also enabled an inference API for you, you just need to add a text-to-image :-) )