|
--- |
|
license: other |
|
language: |
|
- en |
|
base_model: |
|
- black-forest-labs/FLUX.1-dev |
|
pipeline_tag: text-to-image |
|
tags: |
|
- diffusers |
|
- controlnet |
|
- Flux |
|
- image-generation |
|
--- |
|
|
|
# Description |
|
This repository provides a Diffusers version of FLUX.1-dev Hed ControlNet checkpoint by Xlabs AI, [original repo](https://huggingface.co/XLabs-AI/flux-controlnet-hed-v3). |
|
|
|
 |
|
|
|
# How to use |
|
This model can be used directly with the diffusers library |
|
|
|
``` |
|
import torch |
|
from diffusers.utils import load_image |
|
from diffusers import FluxControlNetModel |
|
from diffusers.pipelines import FluxControlNetPipeline |
|
from PIL import Image |
|
import numpy as np |
|
|
|
generator = torch.Generator(device="cuda").manual_seed(87544357) |
|
|
|
controlnet = FluxControlNetModel.from_pretrained( |
|
"Xlabs-AI/flux-controlnet-hed-diffusers", |
|
torch_dtype=torch.bfloat16, |
|
use_safetensors=True, |
|
) |
|
pipe = FluxControlNetPipeline.from_pretrained( |
|
"black-forest-labs/FLUX.1-dev", |
|
controlnet=controlnet, |
|
torch_dtype=torch.bfloat16 |
|
) |
|
pipe.to("cuda") |
|
|
|
control_image = load_image("https://huggingface.co/Xlabs-AI/flux-controlnet-hed-diffusers/resolve/main/hed_example.png") |
|
prompt = "photo of woman in the cyberpank city" |
|
|
|
image = pipe( |
|
prompt, |
|
control_image=control_image, |
|
controlnet_conditioning_scale=0.7, |
|
num_inference_steps=25, |
|
guidance_scale=3.5, |
|
height=1376, |
|
width=1024, |
|
generator=generator, |
|
num_images_per_prompt=1, |
|
).images[0] |
|
|
|
image.save("output_test_controlnet.png") |
|
``` |
|
|
|
## License |
|
|
|
Our weights fall under the [FLUX.1 [dev]](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md) Non-Commercial License<br/> |
