metadata
base_model: stabilityai/stable-diffusion-xl-base-1.0
library_name: diffusers
license: openrail++
tags:
- text-to-image
- text-to-image
- diffusers-training
- diffusers
- lora
- template:sd-lora
- stable-diffusion-xl
- stable-diffusion-xl-diffusers
datasets:
- data-is-better-together/open-image-preferences-v1-binarized
language:
- en
pipeline_tag: text-to-image
Low Rank Adapted Supervised Fine Tuned Stable Diffusion XL
Comparison
| Prompt | SDXL | Fine Tuned |
|---|---|---|
| a boat in the canals of Venice, painted in gouache with soft, flowing brushstrokes and vibrant, translucent colors, capturing the serene reflection on the water under a misty ambiance, with rich textures and a dynamic perspective |  |
 |
| Grainy shot of a robot cooking in the kitchen, with soft shadows and nostalgic film texture. |  |
 |
Model description
These are ariG23498/open-image-preferences-v1-sdxl-lora LoRA adaption weights for stabilityai/stable-diffusion-xl-base-1.0.
The weights were trained using DreamBooth using the open-image-preferences-v1-binarized dataset.
Use with diffusers
from diffusers import AutoPipelineForText2Image
import torch
pipeline = AutoPipelineForText2Image.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0",
torch_dtype=torch.bfloat16
).to('cuda')
pipeline.load_lora_weights('ariG23498/open-image-preferences-v1-sdxl-lora', weight_name='pytorch_lora_weights.safetensors')
prompt = "ENTER PROMPT"
image = pipeline(prompt).images[0]
Command to train the model
!accelerate launch examples/dreambooth/train_dreambooth_lora_sdxl.py \
--pretrained_model_name_or_path "stabilityai/stable-diffusion-xl-base-1.0" \
--dataset_name "data-is-better-together/open-image-preferences-v1-binarized" \
--hub_model_id "ariG23498/open-image-preferences-v1-sdxl-lora" \
--push_to_hub \
--output_dir "open-image-preferences-v1-sdxl-lora" \
--image_column "chosen" \
--caption_column "prompt" \
--mixed_precision="bf16" \
--resolution=1024 \
--train_batch_size=1 \
--repeats=1 \
--report_to="wandb"\
--gradient_accumulation_steps=1 \
--gradient_checkpointing \
--learning_rate=1.0 \
--text_encoder_lr=1.0 \
--optimizer="prodigy"\
--lr_scheduler="constant" \
--lr_warmup_steps=0 \
--rank=8 \
--checkpointing_steps=2000 \
--seed="0"