Upload folder using huggingface_hub
Browse files- README.md +77 -0
- embeddings.pti +0 -0
- lora.safetensors +3 -0
- special_params.json +1 -0
README.md
ADDED
@@ -0,0 +1,77 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: creativeml-openrail-m
|
3 |
+
tags:
|
4 |
+
- text-to-image
|
5 |
+
- stable-diffusion
|
6 |
+
- lora
|
7 |
+
- diffusers
|
8 |
+
base_model: stabilityai/stable-diffusion-xl-base-1.0
|
9 |
+
pivotal_tuning: true
|
10 |
+
textual_embeddings: embeddings.pti
|
11 |
+
instance_prompt: <s0><s1>
|
12 |
+
inference: false
|
13 |
+
---
|
14 |
+
# sdxl-tng-interior LoRA by [fofr](https://replicate.com/fofr)
|
15 |
+
### SDXL fine-tune of Star Trek Next Generation interiors
|
16 |
+
|
17 |
+
![lora_image](https://replicate.delivery/pbxt/CGxE1DgG675SMNPUH8NAuTdHkEh3Cw3l78ze4XbR52f1wWeiA/out-0.png)
|
18 |
+
>
|
19 |
+
|
20 |
+
## Inference with Replicate API
|
21 |
+
Grab your replicate token [here](https://replicate.com/account)
|
22 |
+
```bash
|
23 |
+
pip install replicate
|
24 |
+
export REPLICATE_API_TOKEN=r8_*************************************
|
25 |
+
```
|
26 |
+
|
27 |
+
```py
|
28 |
+
import replicate
|
29 |
+
|
30 |
+
output = replicate.run(
|
31 |
+
"sdxl-tng-interior@sha256:45f1d0cf3445f54d4b19a2a03e53b15abd7237ea72e2fb4824b193ffa429e31f",
|
32 |
+
input={"prompt": "A photo in the style of TOK, interior, house - sustainable, minimalist, organic, light-filled, dynamic, efficient, autonomous, connected, harmonious, innovative, detailed, 8k, high resolution, sharp focus"}
|
33 |
+
)
|
34 |
+
print(output)
|
35 |
+
```
|
36 |
+
You may also do inference via the API with Node.js or curl, and locally with COG and Docker, [check out the Replicate API page for this model](https://replicate.com/fofr/sdxl-tng-interior/api)
|
37 |
+
|
38 |
+
## Inference with 🧨 diffusers
|
39 |
+
Replicate SDXL LoRAs are trained with Pivotal Tuning, which combines training a concept via Dreambooth LoRA with training a new token with Textual Inversion.
|
40 |
+
As `diffusers` doesn't yet support textual inversion for SDXL, we will use cog-sdxl `TokenEmbeddingsHandler` class.
|
41 |
+
|
42 |
+
The trigger tokens for your prompt will be `<s0><s1>`
|
43 |
+
|
44 |
+
```shell
|
45 |
+
pip install diffusers transformers accelerate safetensors huggingface_hub
|
46 |
+
git clone https://github.com/replicate/cog-sdxl cog_sdxl
|
47 |
+
```
|
48 |
+
|
49 |
+
```py
|
50 |
+
import torch
|
51 |
+
from huggingface_hub import hf_hub_download
|
52 |
+
from diffusers import DiffusionPipeline
|
53 |
+
from cog_sdxl.dataset_and_utils import TokenEmbeddingsHandler
|
54 |
+
from diffusers.models import AutoencoderKL
|
55 |
+
|
56 |
+
pipe = DiffusionPipeline.from_pretrained(
|
57 |
+
"stabilityai/stable-diffusion-xl-base-1.0",
|
58 |
+
torch_dtype=torch.float16,
|
59 |
+
variant="fp16",
|
60 |
+
).to("cuda")
|
61 |
+
|
62 |
+
pipe.load_lora_weights("fofr/sdxl-tng-interior", weight_name="lora.safetensors")
|
63 |
+
|
64 |
+
text_encoders = [pipe.text_encoder, pipe.text_encoder_2]
|
65 |
+
tokenizers = [pipe.tokenizer, pipe.tokenizer_2]
|
66 |
+
|
67 |
+
embedding_path = hf_hub_download(repo_id="fofr/sdxl-tng-interior", filename="embeddings.pti", repo_type="model")
|
68 |
+
embhandler = TokenEmbeddingsHandler(text_encoders, tokenizers)
|
69 |
+
embhandler.load_embeddings(embedding_path)
|
70 |
+
prompt="A photo in the style of <s0><s1>, interior, house - sustainable, minimalist, organic, light-filled, dynamic, efficient, autonomous, connected, harmonious, innovative, detailed, 8k, high resolution, sharp focus"
|
71 |
+
images = pipe(
|
72 |
+
prompt,
|
73 |
+
cross_attention_kwargs={"scale": 0.8},
|
74 |
+
).images
|
75 |
+
#your output image
|
76 |
+
images[0]
|
77 |
+
```
|
embeddings.pti
ADDED
lora.safetensors
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:d6b781d7c181e8c00c70d45b9f507a088ccbe0f48d6fabcb2bcf274d8e6d693b
|
3 |
+
size 185968776
|
special_params.json
ADDED
@@ -0,0 +1 @@
|
|
|
|
|
1 |
+
{"TOK": "<s0><s1>"}
|