SD 3.5 Large Modern Anime Full Model Card
This is an experimental model. I full-finetune SD 3.5 Large by Quality Tuning only.
Usage
import torch
from diffusers import SD3Transformer2DModel, StableDiffusion3Pipeline
transformer = SD3Transformer2DModel.from_single_file(
"https://huggingface.co/alfredplpl/sd3-5-large-modern-anime-full/blob/main/sd3_5_large_modern_anime_full.safetensors",
torch_dtype=torch.bfloat16,
)
pipe = StableDiffusion3Pipeline.from_pretrained(
"stabilityai/stable-diffusion-3.5-large",
transformer=transformer,
torch_dtype=torch.bfloat16,
)
pipe.enable_model_cpu_offload()
image = pipe("modern anime style, A close-up shot of a girl's face in the center, looking directly at the viewer. Autumn maple trees with red leaves frame both the left and right sides of the background, with the sky visible in the middle.").images[0]
image.save("sd35.png")
How to Make
Prerequisites
- A6000x1 (48GB)
- Private dataset: 3000 images (collected manually)
Procedure
I used sd-scripts. The dataset config as is follows:
[general]
enable_bucket = true # Aspect Ratio Bucketingを使うか否か
[[datasets]]
resolution = 1024 # 学習解像度
batch_size = 4 # バッチサイズ
[[datasets.subsets]]
image_dir = '/mnt/NVM2/manual_now' # 学習用画像を入れたフォルダを指定
metadata_file = 'manual_dcap2.json' # メタデータファイル名
I ran the command:
accelerate launch --num_cpu_threads_per_process 1 sd3_train.py --pretrained_model_name_or_path='/mnt/NVM2/sd3_5/sd3.5_large.safetensors' --output_dir='/mnt/NVM2/sd3_5' --output_name=modern_anime --dataset_config=manual_dcap2.toml --save_model_as=safetensors --learning_rate=5e-6 --sdpa --gradient_checkpointing --mixed_precision=bf16 --full_bf16 --max_train_epochs=10 --min_bucket_reso=512 --max_bucket_reso=2048 --clip_l='/mnt/NVM2/sd3_5/clip_l.safetensors' --clip_g='/mnt/NVM2/sd3_5/clip_g.safetensors' --gradient_accumulation_steps=1 --t5xxl='/mnt/NVM2/sd3_5/t5xxl_fp16.safetensors' --optimizer_type=Lion8bit --cache_text_encoder_outputs_to_disk --cache_text_encoder_outputs --cache_latents --cache_latents_to_disk --save_every_n_epochs=1
Model tree for alfredplpl/sd3-5-large-modern-anime-full
Base model
stabilityai/stable-diffusion-3.5-large