Upload folder using huggingface_hub
Browse files- README.md +77 -0
- embeddings.pti +0 -0
- lora.safetensors +3 -0
- special_params.json +1 -0
README.md
ADDED
@@ -0,0 +1,77 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: creativeml-openrail-m
|
3 |
+
tags:
|
4 |
+
- text-to-image
|
5 |
+
- stable-diffusion
|
6 |
+
- lora
|
7 |
+
- diffusers
|
8 |
+
base_model: stabilityai/stable-diffusion-xl-base-1.0
|
9 |
+
pivotal_tuning: true
|
10 |
+
textual_embeddings: embeddings.pti
|
11 |
+
instance_prompt: <s0><s1>
|
12 |
+
inference: false
|
13 |
+
---
|
14 |
+
# sdxl-emoji LoRA by [fofr](https://replicate.com/fofr)
|
15 |
+
### An SDXL fine-tune based on Apple Emojis
|
16 |
+
|
17 |
+
![lora_image](https://replicate.delivery/pbxt/a3z81v5vwlKfLq1H5uBqpVmkHalOVup0jSLma9E2UaF3tawIA/out-0.png)
|
18 |
+
>
|
19 |
+
|
20 |
+
## Inference with Replicate API
|
21 |
+
Grab your replicate token [here](https://replicate.com/account)
|
22 |
+
```bash
|
23 |
+
pip install replicate
|
24 |
+
export REPLICATE_API_TOKEN=r8_*************************************
|
25 |
+
```
|
26 |
+
|
27 |
+
```py
|
28 |
+
import replicate
|
29 |
+
|
30 |
+
output = replicate.run(
|
31 |
+
"sdxl-emoji@sha256:dee76b5afde21b0f01ed7925f0665b7e879c50ee718c5f78a9d38e04d523cc5e",
|
32 |
+
input={"prompt": "A TOK emoji of a man"}
|
33 |
+
)
|
34 |
+
print(output)
|
35 |
+
```
|
36 |
+
You may also do inference via the API with Node.js or curl, and locally with COG and Docker, [check out the Replicate API page for this model](https://replicate.com/fofr/sdxl-emoji/api)
|
37 |
+
|
38 |
+
## Inference with 🧨 diffusers
|
39 |
+
Replicate SDXL LoRAs are trained with Pivotal Tuning, which combines training a concept via Dreambooth LoRA with training a new token with Textual Inversion.
|
40 |
+
As `diffusers` doesn't yet support textual inversion for SDXL, we will use cog-sdxl `TokenEmbeddingsHandler` class.
|
41 |
+
|
42 |
+
The trigger tokens for your prompt will be `<s0><s1>`
|
43 |
+
|
44 |
+
```shell
|
45 |
+
pip install diffusers transformers accelerate safetensors huggingface_hub
|
46 |
+
git clone https://github.com/replicate/cog-sdxl cog_sdxl
|
47 |
+
```
|
48 |
+
|
49 |
+
```py
|
50 |
+
import torch
|
51 |
+
from huggingface_hub import hf_hub_download
|
52 |
+
from diffusers import DiffusionPipeline
|
53 |
+
from cog_sdxl.dataset_and_utils import TokenEmbeddingsHandler
|
54 |
+
from diffusers.models import AutoencoderKL
|
55 |
+
|
56 |
+
pipe = DiffusionPipeline.from_pretrained(
|
57 |
+
"stabilityai/stable-diffusion-xl-base-1.0",
|
58 |
+
torch_dtype=torch.float16,
|
59 |
+
variant="fp16",
|
60 |
+
).to("cuda")
|
61 |
+
|
62 |
+
pipe.load_lora_weights("fofr/sdxl-emoji", weight_name="lora.safetensors")
|
63 |
+
|
64 |
+
text_encoders = [pipe.text_encoder, pipe.text_encoder_2]
|
65 |
+
tokenizers = [pipe.tokenizer, pipe.tokenizer_2]
|
66 |
+
|
67 |
+
embedding_path = hf_hub_download(repo_id="fofr/sdxl-emoji", filename="embeddings.pti", repo_type="model")
|
68 |
+
embhandler = TokenEmbeddingsHandler(text_encoders, tokenizers)
|
69 |
+
embhandler.load_embeddings(embedding_path)
|
70 |
+
prompt="A <s0><s1> emoji of a man"
|
71 |
+
images = pipe(
|
72 |
+
prompt,
|
73 |
+
cross_attention_kwargs={"scale": 0.8},
|
74 |
+
).images
|
75 |
+
#your output image
|
76 |
+
images[0]
|
77 |
+
```
|
embeddings.pti
ADDED
lora.safetensors
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:5934fe70ec5f06be145670315a4cd20334abb51ace38bee4f9265fc82d035f54
|
3 |
+
size 185968776
|
special_params.json
ADDED
@@ -0,0 +1 @@
|
|
|
|
|
1 |
+
{"TOK": "<s0><s1>"}
|