File size: 2,764 Bytes
4bc4845 d4af770 7bc50e8 10442db d4af770 4bc4845 ac7275a d4af770 597ad97 d4af770 597ad97 d4af770 597ad97 d4af770 139b847 d4af770 10442db d4af770 a260c1a ac7275a 4bc4845 d4af770 10442db d4af770 4bc4845 10442db d4af770 10442db 4bc4845 d4af770 d8c735b d4af770 4bc4845 d8c735b d4af770 4bc4845 d4af770 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 |
---
library_name: diffusers
base_model: runwayml/stable-diffusion-v1-5
tags:
- lora
- text-to-image
license: openrail++
inference: false
---
# Latent Consistency Model (LCM) LoRA: SDv1-5
Latent Consistency Model (LCM) LoRA was proposed in [LCM-LoRA: A universal Stable-Diffusion Acceleration Module](TODO:)
by *Simian Luo, Yiqin Tan, Suraj Patil, Daniel Gu et al.*
It is a distilled consistency adapter for [`runwayml/stable-diffusion-v1-5`](https://huggingface.co/runwayml/stable-diffusion-v1-5) that allows
to reduce the number of inference steps to only between **2 - 8 steps**.
| Model | Params / M |
|----------------------------------------------------------------------------|------------|
| [**lcm-lora-sdv1-5**](https://huggingface.co/latent-consistency/lcm-lora-sdv1-5) | **67.5** |
| [lcm-lora-ssd-1b](https://huggingface.co/latent-consistency/lcm-lora-ssd-1b) | 105 |
| [lcm-lora-sdxl](https://huggingface.co/latent-consistency/lcm-lora-sdxl) | 197M |
## Usage
LCM-LoRA is supported in 🤗 Hugging Face Diffusers library from version v0.23.0 onwards. To run the model, first
install the latest version of the Diffusers library as well as `peft`, `accelerate` and `transformers`.
audio dataset from the Hugging Face Hub:
```bash
pip install --upgrade pip
pip install --upgrade diffusers transformers accelerate peft
```
### Text-to-Image
The adapter can be loaded with SDv1-5 or deviratives. Here we use [`Lykon/dreamshaper-7`](https://huggingface.co/Lykon/dreamshaper-7). Next, the scheduler needs to be changed to [`LCMScheduler`](https://huggingface.co/docs/diffusers/v0.22.3/en/api/schedulers/lcm#diffusers.LCMScheduler) and we can reduce the number of inference steps to just 2 to 8 steps.
Please make sure to either disable `guidance_scale` or use values between 1.0 and 2.0.
```python
import torch
from diffusers import LCMScheduler, AutoPipelineForText2Image
model_id = "Lykon/dreamshaper-7"
adapter_id = "latent-consistency/lcm-lora-sdv1-5"
pipe = AutoPipelineForText2Image.from_pretrained(model_id, torch_dtype=torch.float16, variant="fp16")
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)
pipe.to("cuda")
# load and fuse lcm lora
pipe.load_lora_weights(adapter_id)
pipe.fuse_lora()
prompt = "Self-portrait oil painting, a beautiful cyborg with golden hair, 8k"
# disable guidance_scale by passing 0
image = pipe(prompt=prompt, num_inference_steps=4, guidance_scale=0).images[0]
```
### Image-to-Image
Works as well! TODO docs
### Inpainting
Works as well! TODO docs
### ControlNet
Works as well! TODO docs
### T2I Adapter
Works as well! TODO docs
## Speed Benchmark
TODO
## Training
TODO |