--- tags: - text-to-image - sd3 - lora - diffusers - template:sd-lora - StableDiffusion3 - StableDiffusion3.5 - ai-toolkit widget: - text: >- /@step 4000 weights:/ HST style autochrome photo of a drugged God Christ-insect in wooden pillbox being pinned needle-poked by a giant Marina Tsvetaeva in a long red gown, arcane symbolism, colorful HD realistic film in Yury Annenkov & David Lynch styles, detailed face worrying, full-height red-robed woman roams European city circa 1930 output: url: samples/1729711624663__000004000_4.jpg - text: >- /@step 1600 weights:/HST style autochrome photo of a drugged God Christ-insect in wooden pillbox being pinned needle-poked by a giant Marina Tsvetaeva in a long red gown, arcane symbolism, colorful HD realistic film in Yury Annenkov & David Lynch styles, detailed face worrying, full-height red-robed woman roams European city circa 1930 output: url: samples/1729700659095__000001600_4.jpg - text: >- /@step 1800 weights:/HST style autochrome photo of a drugged God Christ-insect in wooden pillbox being pinned needle-poked by a giant Marina Tsvetaeva in a long red gown, arcane symbolism, colorful HD realistic film in Yury Annenkov & David Lynch styles, detailed face worrying, full-height red-robed woman roams European city circa 1930 output: url: samples/1729701571749__000001800_4.jpg - text: >- /@step 800 weights:/HHST style autochrome photo of a drugged God Christ-insect in wooden pillbox being pinned needle-poked by a giant Marina Tsvetaeva in a long red gown, arcane symbolism, colorful HD realistic film in Yury Annenkov & David Lynch styles, detailed face worrying, full-height red-robed woman roams European city circa 1930 output: url: samples/1729696985671__000000800_4.jpg - text: >- /@step 1800 weights:/HST style communist poster with text "JOIN RCA!", over autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich, dancing with red feathered drunken dinosaur, an early conceptual artist. Lenin is full of contageous awe, his blemished skin flushing with anxious excitement, his famous bald spot sweatily glistening under warm lights. In the back, Krupskaya and Inessa Armand laugh. output: url: samples/1729701488064__000001800_0.jpg - text: >- /@step 4000 weights:/HHST style communist poster with text "JOIN RCA!", over autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich, dancing with red feathered drunken dinosaur, an early conceptual artist. Lenin is full of contageous awe, his blemished skin flushing with anxious excitement, his famous bald spot sweatily glistening under warm lights. In the back, Krupskaya and Inessa Armand laugh. output: url: samples/1729711540947__000004000_0.jpg - text: >- /@step 1600 weights:/ HST style communist poster with text "JOIN RCA!", over autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich, dancing with red feathered drunken dinosaur, an early conceptual artist. Lenin is full of contageous awe, his blemished skin flushing with anxious excitement, his famous bald spot sweatily glistening under warm lights. In the back, Krupskaya and Inessa Armand laugh. output: url: samples/1729700575417__000001600_0.jpg - text: >- /@step 1600 weights:/ HST autochrome style analog dslr award-winning 8k art photo showing a nurse battling a giant cell phone, above text caption of \ONCE WE WERE ALL HARRY GARDNER!\, in a highly realistic modern American medical hospital output: url: samples/1729700596338__000001600_1.jpg - text: >- /@step 1000 weights:/ HST autochrome style analog dslr award-winning 8k art photo showing a nurse battling a giant cell phone, above text caption of \ONCE WE WERE ALL HARRY GARDNER!\, in a highly realistic modern American medical hospital output: url: samples/1729697845170__000001000_1.jpg - text: >- /@step 3600 weights:/ HST style autochrome photograph of a dark CIA agent Koala leaping at the throat of goth freedom-fighter Julian Assange, tragic atmosphere, award-winning art photo output: url: samples/1729709764758__000003600_2.jpg - text: >- /@step 1200 weights:/ HST style autochrome photograph of a dark CIA agent Koala leaping at the throat of goth freedom-fighter Julian Assange, tragic atmosphere, award-winning art photo output: url: samples/1729698783304__000001200_2.jpg - text: >- /@step 1000 weights:/ HST style autochrome photograph of a dark CIA agent Koala leaping at the throat of goth freedom-fighter Julian Assange, tragic atmosphere, award-winning art photo output: url: samples/1729697866093__000001000_2.jpg - text: >- /@step 4000 weights:/ HST style autochrome photo of realistic green-eyed black cat, with prominent regions of white fur, playing a piano and singing, amateur 2004 photograph shot on a cell phone in a Los Angeles apartment kitchen output: url: samples/1729711603742__000004000_3.jpg - text: >- /@step 3000 weights:/ HST style autochrome photo of realistic green-eyed black cat, with prominent regions of white fur, playing a piano and singing, amateur 2004 photograph shot on a cell phone in a Los Angeles apartment kitchen output: url: samples/1729707029633__000003000_3.jpg - text: >- /@step 1800 weights:/ HST style autochrome photo of realistic green-eyed black cat, with prominent regions of white fur, playing a piano and singing, amateur 2004 photograph shot on a cell phone in a Los Angeles apartment kitchen output: url: samples/1729701550832__000001800_3.jpg - text: >- /@step 1000 weights:/ HST style autochrome photo of realistic green-eyed black cat, with prominent regions of white fur, playing a piano and singing, amateur 2004 photograph shot on a cell phone in a Los Angeles apartment kitchen output: url: samples/1729697887015__000001000_3.jpg base_model: stabilityai/stable-diffusion-3.5-large license: creativeml-openrail-m language: - en pipeline_tag: text-to-image library_name: diffusers --- # HSTsd3ii Model trained with [AI Toolkit by Ostris](https://github.com/ostris/ai-toolkit) ## Trigger words 'HST style autochrome photo' ## Config Parameters *Dim:16 Alpha:32 Optimizer:Ademamix8bit LR:2e-4* **More info below!**
Fine-tuned using the **Google Colab Notebook*** Version of **ai-toolkit**.
I've used A100 via Colab Pro. However, training SD3.5 may potentially work with Free Colab or lower Vram in general:
Especially if one were to use:
...Say, *lower rank (try 4 or 8), dataset size (in terms of caching/bucketing/pre-loading impacts), 1 batch size, Adamw8bit optimizer, 512 resolution, maybe adding the /lowvram, true/ argument, and plausibly specifying alternate quantization variants.*
Generally, VRAM expenditures tend to be lower than for Flux during training. So, try it! I certainly will.
**To use on Colab**, modify a Flux template Notebook from [here](https://github.com/ostris/ai-toolkit/tree/main/notebooks) with parameters from Ostris' example config for SD3.5 [here](https://github.com/ostris/ai-toolkit/blob/main/config/examples/train_lora_sd35_large_24gb.yaml)! ``` job: extension config: name: HSTsd3ii process: - type: sd_trainer training_folder: /content/drive/MyDrive/HSTsd3ii performance_log_every: 600 device: cuda:0 network: type: lora linear: 16 linear_alpha: 32 save: dtype: float16 save_every: 250 push_to_hub: true hf_repo_id: AlekseyCalvin/HSTsd3iii hf_private: true max_step_saves_to_keep: 16 datasets: - folder_path: /content/dataset caption_ext: txt caption_dropout_rate: 0.0 shuffle_tokens: false cache_latents_to_disk: true resolution: - 1024 train: batch_size: 4 steps: 4000 gradient_accumulation_steps: 1 train_unet: true train_text_encoder: false gradient_checkpointing: true noise_scheduler: flowmatch timestep_type: linear optimizer: ademamix8bit lr: 0.0002 skip_first_sample: true ema_config: use_ema: true ema_decay: 0.8 dtype: bf16 model: name_or_path: stabilityai/stable-diffusion-3.5-large is_v3: true quantize: true ``` Plus validation settings:
Prompts like the above, at 1024, guidance scale 4, 25 steps, seed 42, no negatives. ## Download model and use it with ComfyUI, AUTOMATIC1111, SD.Next, Invoke AI, etc. Weights for this model are available in Safetensors format. [Download](/AlekseyCalvin/HSTsd3iii/tree/main) them in the Files & versions tab. ## Use it with the [🧨 diffusers library](https://github.com/huggingface/diffusers) ```py from diffusers import AutoPipelineForText2Image import torch pipeline = AutoPipelineForText2Image.from_pretrained('stabilityai/stable-diffusion-3.5-large', torch_dtype=torch.float16).to('cuda') pipeline.load_lora_weights('AlekseyCalvin/HSTsd3iii', weight_name='HSTsd3ii.safetensors') image = pipeline('HST style communist poster with text "JOIN RCA!", over autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich, dancing with red feathered drunken dinosaur, an early conceptual artist. Lenin is full of contageous awe, his blemished skin flushing with anxious excitement, his famous bald spot sweatily glistening under warm lights. In the back, Krupskaya and Inessa Armand laugh. ').images[0] image.save("my_image.png") ``` For more details, including weighting, merging and fusing LoRAs, check the [documentation on loading LoRAs in diffusers](https://huggingface.co/docs/diffusers/main/en/using-diffusers/loading_adapters)