How to use a particular safetensors model with Diffusers?
I am loving the model.
But I have a question about how to use the Inpaint .safetensors model with Diffusers...
Calling "Lykon/DreamShaper" will lead to the default, non-inpaint model.
But calling "/home/user/Dreamshaper/DreamShaper_7-INPAINTING.inpainting.safetensors" will give me the error of "huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/home/heran/DreamShaper/DreamShaper_7_INPAINTING.inpainting.safetensors'. Use repo_type argument if needed."
Would really appreciate if anyone can help me with this.
Not sure Lykon made the diffusers version for inpainting yet, i could be wrong.
I didn't, is it needed? Don't script support safetensors nowdays?
I dont think you need to import Inpaint by specifying url.
You could just pass name Lykon/Dreamshaper to StablefdiffusionInpaintPipeline and it should work.
Or you could just unpack the components of main model you have already loaded.
I use the Later, Try this 🐍python code , this worked out for me. :)
from diffusers import DiffusionPipeline
from diffusers import StableDiffusionImg2ImgPipeline
from diffusers import StableDiffusionInpaintPipeline
GenerationPipe = DiffusionPipeline.from_pretrained( GenerationModel, torch_dtype=torch.float16 ).to( Device )
VariationPipe = StableDiffusionImg2ImgPipeline.from_pretrained( GenerationModel, **GenerationPipe.components ).to( Device )
InpaintPipe = StableDiffusionInpaintPipeline.from_pretrained( GenerationModel, **GenerationPipe.components ).to( Device )
GeneratedImage = GenerationPipe( Prompt ).images[0]
VariedImage = VariationPipe( VarPrompt, image= GeneratedImage ).images[0]
InpaintedImage = imageinp=InpaintPipe( InpaintPrompt, image=GeneratedImage, mask_image=ImageMask ).images[0]
⚠️ Make sure that you set GenerationModel = 'Lykon/DreamShaper' and Device='cuda'.🤗 I Hope it helps .
@Mohdzaid18 I think @hzhou17 was talking about the inpainting model specifically, which is this other file https://huggingface.co/Lykon/DreamShaper/blob/main/DreamShaper_7-INPAINTING.inpainting.safetensors
Not to be confused with the main model here https://huggingface.co/Lykon/DreamShaper/blob/main/DreamShaper_7_pruned.safetensors
Or with the diffusers (which are the same as the main model)
So which model does StableDiffusionInpaintPipeline uses? Dreamshaper 7 or Dreamshaper_7-INPAINTING
Because I have used StableDiffusionInpaintPipeline and it works perfectly fine for inpainting purposes.
So i assumed it uses Dreamshaper_7-INPAINTING or any other of which version is set to default.
Using the above code that i had provided, iwas able to first generate the image and then provide the mask and rightmost image is the output from StableDiffusionInpaintPipeline, i really think it uses inpaint model to achive this.
Sorry, I didn't quite understand the solution. Does it mean it will automatically select the inpaint safe tensor to perform inpainting?