README.md

---
tags:
- text-to-image
- sd3
- lora
- diffusers
- template:sd-lora
- StableDiffusion3
- StableDiffusion3.5
- ai-toolkit
widget:
- text: >-
    /@step 4000 weights:/ HST style autochrome photo of a drugged God
    Christ-insect in wooden pillbox being pinned needle-poked by a giant Marina
    Tsvetaeva in a long red gown, arcane symbolism, colorful HD realistic film
    in Yury Annenkov & David Lynch styles, detailed face worrying, full-height
    red-robed woman roams European city circa 1930
  output:
    url: samples/1729711624663__000004000_4.jpg
- text: >-
    /@step 1600 weights:/HST style autochrome photo of a drugged God
    Christ-insect in wooden pillbox being pinned needle-poked by a giant Marina
    Tsvetaeva in a long red gown, arcane symbolism, colorful HD realistic film
    in Yury Annenkov & David Lynch styles, detailed face worrying, full-height
    red-robed woman roams European city circa 1930
  output:
    url: samples/1729700659095__000001600_4.jpg
- text: >-
    /@step 1800 weights:/HST style autochrome photo of a drugged God
    Christ-insect in wooden pillbox being pinned needle-poked by a giant Marina
    Tsvetaeva in a long red gown, arcane symbolism, colorful HD realistic film
    in Yury Annenkov & David Lynch styles, detailed face worrying, full-height
    red-robed woman roams European city circa 1930
  output:
    url: samples/1729701571749__000001800_4.jpg
- text: >-
    /@step 800 weights:/HHST style autochrome photo of a drugged God
    Christ-insect in wooden pillbox being pinned needle-poked by a giant Marina
    Tsvetaeva in a long red gown, arcane symbolism, colorful HD realistic film
    in Yury Annenkov & David Lynch styles, detailed face worrying, full-height
    red-robed woman roams European city circa 1930
  output:
    url: samples/1729696985671__000000800_4.jpg
- text: >-
    /@step 1800 weights:/HST style communist poster with text "JOIN RCA!", over
    autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich, 
    dancing with red feathered drunken dinosaur, an early conceptual artist.
    Lenin is full of contageous awe, his blemished skin flushing  with anxious
    excitement, his famous bald spot sweatily glistening under warm lights. In
    the back, Krupskaya and Inessa Armand laugh. 
  output:
    url: samples/1729701488064__000001800_0.jpg
- text: >-
    /@step 4000 weights:/HHST style communist poster with text "JOIN RCA!", over
    autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich, 
    dancing with red feathered drunken dinosaur, an early conceptual artist.
    Lenin is full of contageous awe, his blemished skin flushing  with anxious
    excitement, his famous bald spot sweatily glistening under warm lights. In
    the back, Krupskaya and Inessa Armand laugh. 
  output:
    url: samples/1729711540947__000004000_0.jpg
- text: >-
    /@step 1600 weights:/ HST style communist poster with text "JOIN RCA!", over
    autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich, 
    dancing with red feathered drunken dinosaur, an early conceptual artist.
    Lenin is full of contageous awe, his blemished skin flushing  with anxious
    excitement, his famous bald spot sweatily glistening under warm lights. In
    the back, Krupskaya and Inessa Armand laugh. 
  output:
    url: samples/1729700575417__000001600_0.jpg
- text: >-
    /@step 1600 weights:/ HST autochrome style analog dslr award-winning 8k art
    photo showing a nurse battling a giant cell phone, above text caption of
    \ONCE WE WERE ALL HARRY GARDNER!\, in a highly realistic modern American
    medical hospital 
  output:
    url: samples/1729700596338__000001600_1.jpg
- text: >-
    /@step 1000 weights:/ HST autochrome style analog dslr award-winning 8k art
    photo showing a nurse battling a giant cell phone, above text caption of
    \ONCE WE WERE ALL HARRY GARDNER!\, in a highly realistic modern American
    medical hospital 
  output:
    url: samples/1729697845170__000001000_1.jpg
- text: >-
    /@step 3600 weights:/ HST style autochrome photograph of a dark CIA agent
    Koala leaping at the throat of goth freedom-fighter Julian Assange, tragic
    atmosphere, award-winning art photo
  output:
    url: samples/1729709764758__000003600_2.jpg
- text: >-
    /@step 1200 weights:/ HST style autochrome photograph of a dark CIA agent
    Koala leaping at the throat of goth freedom-fighter Julian Assange, tragic
    atmosphere, award-winning art photo
  output:
    url: samples/1729698783304__000001200_2.jpg
- text: >-
    /@step 1000 weights:/ HST style autochrome photograph of a dark CIA agent
    Koala leaping at the throat of goth freedom-fighter Julian Assange, tragic
    atmosphere, award-winning art photo
  output:
    url: samples/1729697866093__000001000_2.jpg
- text: >-
    /@step 4000 weights:/ HST style autochrome photo of realistic green-eyed
    black cat, with prominent regions of white fur, playing a piano and singing,
    amateur 2004 photograph shot on a cell phone in a Los Angeles apartment
    kitchen
  output:
    url: samples/1729711603742__000004000_3.jpg
- text: >-
    /@step 3000 weights:/ HST style autochrome photo of realistic green-eyed
    black cat, with prominent regions of white fur, playing a piano and singing,
    amateur 2004 photograph shot on a cell phone in a Los Angeles apartment
    kitchen
  output:
    url: samples/1729707029633__000003000_3.jpg
- text: >-
    /@step 1800 weights:/ HST style autochrome photo of realistic green-eyed
    black cat, with prominent regions of white fur, playing a piano and singing,
    amateur 2004 photograph shot on a cell phone in a Los Angeles apartment
    kitchen
  output:
    url: samples/1729701550832__000001800_3.jpg
- text: >-
    /@step 1000 weights:/ HST style autochrome photo of realistic green-eyed
    black cat, with prominent regions of white fur, playing a piano and singing,
    amateur 2004 photograph shot on a cell phone in a Los Angeles apartment
    kitchen
  output:
    url: samples/1729697887015__000001000_3.jpg
base_model: stabilityai/stable-diffusion-3.5-large
license: creativeml-openrail-m
language:
- en
pipeline_tag: text-to-image
library_name: diffusers
---

# HSTsd3ii
Model trained with [AI Toolkit by Ostris](https://github.com/ostris/ai-toolkit)
<Gallery />

## Trigger words

**'HST style autochrome photo'**

## Config Parameters
*Dim:16 Alpha:32 Optimizer:Ademamix8bit LR:2e-4* **More info below!** <br>
Fine-tuned using the **Google Colab Notebook*** of **ai-toolkit**.<br>
I've used A100 via Colab Pro.
However, training SD3.5 may potentially work with Free Colab or lower VRAM in general:<br> 
Especially if one were to use:<br> ...Say, *lower rank (try 4 or 8), dataset size (in terms of caching/bucketing/pre-loading impacts), 1 batch size, Adamw8bit optimizer, 512 resolution, maybe adding the /lowvram, true/ argument, and plausibly specifying alternate quantization variants.* <br>
Generally, VRAM expenditures for fine-tuning SD3.5 tend to be lower than for Flux during training.<br> 
So, try it!<br>
**To use on Colab**, modify a Flux template Notebook from [here](https://github.com/ostris/ai-toolkit/tree/main/notebooks) with parameters from Ostris' example config for SD3.5 [here](https://github.com/ostris/ai-toolkit/blob/main/config/examples/train_lora_sd35_large_24gb.yaml)!
```
job: extension
config:
  name: HSTsd3ii
  process:
  - type: sd_trainer
    training_folder: /content/drive/MyDrive/HSTsd3ii
    performance_log_every: 600
    device: cuda:0
    network:
      type: lora
      linear: 16
      linear_alpha: 32
    save:
      dtype: float16
      save_every: 250
      push_to_hub: true
      hf_repo_id: AlekseyCalvin/HSTsd3iii
      hf_private: true
      max_step_saves_to_keep: 16
    datasets:
    - folder_path: /content/dataset
      caption_ext: txt
      caption_dropout_rate: 0.0
      shuffle_tokens: false
      cache_latents_to_disk: true
      resolution:
      - 1024
    train:
      batch_size: 4
      steps: 4000
      gradient_accumulation_steps: 1
      train_unet: true
      train_text_encoder: false
      gradient_checkpointing: true
      noise_scheduler: flowmatch
      timestep_type: linear
      optimizer: ademamix8bit
      lr: 0.0002
      skip_first_sample: true
      ema_config:
        use_ema: true
        ema_decay: 0.8
      dtype: bf16
    model:
      name_or_path: stabilityai/stable-diffusion-3.5-large
      is_v3: true
      quantize: true
```
Plus validation settings:<br> Prompts like the above, at 1024, guidance scale 4, 25 steps, seed 42, no negatives.

## Download model and use it with ComfyUI, AUTOMATIC1111, SD.Next, Invoke AI, etc.

Weights for this model are available in Safetensors format.

[Download](/AlekseyCalvin/HSTsd3iii/tree/main) them in the Files & versions tab.

## Use it with the [🧨 diffusers library](https://github.com/huggingface/diffusers)

```py
from diffusers import AutoPipelineForText2Image
import torch

pipeline = AutoPipelineForText2Image.from_pretrained('stabilityai/stable-diffusion-3.5-large', torch_dtype=torch.float16).to('cuda')
pipeline.load_lora_weights('AlekseyCalvin/HSTsd3iii', weight_name='HSTsd3ii.safetensors')
image = pipeline('HST style communist poster with text "JOIN RCA!", over autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich, dancing with red feathered drunken dinosaur, an early conceptual artist. Lenin is full of contageous awe, his blemished skin flushing with anxious excitement, his famous bald spot sweatily glistening under warm lights. In the back, Krupskaya and Inessa Armand laugh. ').images[0]
image.save("my_image.png")
```

For more details, including weighting, merging and fusing LoRAs, check the [documentation on loading LoRAs in diffusers](https://huggingface.co/docs/diffusers/main/en/using-diffusers/loading_adapters)