adds retrained Hibiki LoRA

README.md

@@ -23,7 +23,9 @@ Here you will find the various LoRAs I've trained, typically of Blue Archive cha
 [![Chise](chise/chara-chise-v2.png)](https://huggingface.co/khanon/lora-training/blob/main/chise/README.md)
 
 ### Hibiki
-[Available on old Mega.co.nz repository.](https://mega.nz/folder/SqYwQTRI#GN2SmGTBsV6S4q-L-V4VeA)
+[Nekozuka Hibiki / 猫塚ヒビキ / 네코즈카 히비키 / 猫塚响](https://huggingface.co/khanon/lora-training/blob/main/hibiki/README.md)
+
+[![Hibiki](hibiki/chara-hibiki-v3.png)](https://huggingface.co/khanon/lora-training/blob/main/hibiki/README.md)
 
 ### Hina
 [Available on old Mega.co.nz repository.](https://mega.nz/folder/SqYwQTRI#GN2SmGTBsV6S4q-L-V4VeA)

hibiki/README.md

@@ -1,38 +1,61 @@
 # Nekozuka Hibiki (Blue Archive)
+猫塚ヒビキ (ブルーアーカイブ) / 네코즈카 히비키 (블루 아카이브) / 猫塚响 (碧蓝档案)
+
+[**Download here.**](chara-hibiki-v3.safetensors)
+
+## Table of Contents
+- [Preview](#preview)
+- [Usage](#usage)
+- [Training](#training)
+- [Revisions](#revisions)
+
+## Preview
+![Hibiki portrait](chara-hibiki-v3.png)
+![Hibiki 1](example-001-v3.png)
+![Hibiki 2](example-002-v3.png)
+![Hibiki 3](example-003-v3.png)
 
 ## Usage
-*Important:* This is a fairly temperamental LoRA due to the dataset, and it needs some wrangling to get good results.  It won't look good with vanilla NAI and the standard NAI negative prompt, despite being trained on nai-animefull-final.
-- Use a strong negative prompt.  Consider using the bad_prompt_v2 embed at a reduced strength to dramatically improve things, though it does affect the style somewhat.
-- Use a strong model.  AbyssOrangeMix2 and nutmegmix work well with this LoRA.
-- Use the negative prompt liberally to suppress cheerleader Hibiki if you don't want her, otherwise her traits tend to take over.
+_※ Hibiki's outfits are rather complicated, so I have documented all of her tags here for your reference.  It may not be necessary to use all of them._
 
-To summon Hibiki, use the following tags.  Adjust strength as needed.
-  - `1girl, halo, black hair, blue eyes, bright pupils, animal ears, dog girl, tail, hair bobbles, goggles, eyewear on head, medium breasts`
+Use any or all of these tags to summon Hibiki: `hibiki, halo, dog ears, bright pupils, goggles on head, black hair, dog tail`
 
-For regular Hibiki, add the following tags.  Adjust strength as needed.
-  - Prompt: `(black camisole, jacket, black shorts:1.2), (fishnet legwear:1.1)`
-  - Negative prompt: `(midriff, navel, skin tight, tight:1.4), (tattoo, arm tattoo, star sticker:1.30), ski goggles, wavy mouth, embarrassed`
-    - Cheerleader Hibiki tends to have a permanent embarrassed/wavy mouth expression unless you use negative tags to get rid of it.
+For her normal outfit: `twintails, hair bobbles, engineering goggles, purple choker, multicolored jacket, black camisole, grey shirt, black shorts, fishnet pantyhose, dog tags`
 
-For cheerleader Hibiki, add the following tags.
-  - Prompt: `cheerleader, midriff, embarrassed, wavy mouth, crop top, white pleated skirt`
-  - Negative prompt: `fishnets`
+For her cheerleader alt: `cheerleader, ponytail, crop top, wing collar, criss-cross halter, pleated miniskirt, pom pom (cheerleading)`
+- Use `cheerleader` to strengthen or weaken the effect.
+- If the stickers appear on incorrect outfits, add `star sticker, sticker on face, sticker on arm` to the negative prompt.
+- If her name tag and collar appear on incorrect outfits, add `wing collar` to the negative prompt.
 
-You can add or ignore `hibiki, blue archive`; while they were in her captions, they don't have an especially strong effect.
+Calm expression: `light smile, expressionless`
 
-Adjust the weight as needed.  Weights from 0.95 up to 1.25 work well (higher weights may summon Cheerleader Hibiki unintentionally).
+Embarrassed expression: `blush, embarrassed, wavy mouth, open mouth`
+
+Weapon: `mortar (weapon), rocket, mortar shell, suitcase, m logo`
+
+Weight 0.9 - 1.0 should be ideal.  The LoRA is slightly overfit, I'll probably retrain it with slightly fewer steps in the future.  It's still a substantial improvement over the previous version and is reasonably flexible with Hibiki's outfit.
 
 ## Training
-*All parameters are provided in the accompanying JSON files.*
-- Trained on 119 images, separated into two sets.
-  - Hibiki (regular) -- 54 images, 10 repeats
-  - Hibiki (cheerleader) -- 65 images, 6 repeats
-  - Dataset included a mixture of SFW and NSFW.  Mostly SFW.
-  - As you can guess, her cheerleader alt makes up the vast majority of her art, and artists are more consistent when drawing her. Training them all together did not work well, so I had to split up the datasets.
-- Dataset was tagged with WD1.4 interrogator.  Shuffling was enabled.
-  - `hibiki, blue archive` were added to the start of each caption.  keep_tokens=2 was set to prevent shuffling those tokens.
-  - Her cheerleader outfit is much more easily recognized by the tagger, leading to stronger tags.  Even human artists can't seem to agree on what she's wearing in her normal outfit.
-- Trained at 768px resolution.  I stopped training a 512 variant because it was almost always worse and added to training time.
-- Two variants included.
-  - v1: first attempt at splitting the dataset.  Works well, but not as coherent in some ways (halo particularly) and still tends to blend details from her two outfits.
-  - v2: using the split dataset. Increased batch size slightly (3 >>> 4), reduced learning rate (3e-5 >>> 1e-5), increased epochs (1 >>> 3).  Generally an improvement, though sometimes v1 is easier to wrangle.
+*Exact parameters are provided in the accompanying JSON files.*
+- Trained on a set of 162 images; 82 cheerleader, 80 normal.
+  - 8 repeats for normal outfit
+  - 7 repeats for cheerleader outfit
+  - 3 batch size, 4 epochs
+  - `(82*7 + 80*8) / 3 * 4` = 1619 steps
+- 0.0905 loss
+- 832x832 training resolution
+- `constant_with_warmup` scheduler
+- Initially tagged with WD1.4 swin-v2 model, then heavily edited
+- Used network_dimension 128 (same as usual) / network alpha 128 (default)
+  - Resized to 32dim after training
+- Trained without VAE.
+- [Training dataset available here.](https://mega.nz/folder/2u4HgRoK#1wqHcDAJRi6jrtTAmNjB-Q)
+
+## Revisions
+- v3 (2023-02-13)
+  - Retrained from scratch since the original LoRA was not very good.
+  - Added many new images to the dataset, particularly for Hibiki's regular outfit.
+  - Completely re-tagged the dataset. Used WD1.4 swin-v2 as a starting point, then performed extensive editing and cleanup for accuracy and consistency.  Added many new tags to better describe Hibiki's outfits and expressions.
+  - Uses improved hyperparamters from my more recent models.
+- v1 (2023-01-10)
+  - Initial release.

hibiki/chara-hibiki-v3.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae2f43fa0fc02447ce982e04632c0f2020762392d394823084838c7cc7c901fc
+size 37877571

hibiki/{lora_character_hibiki_119img9repeat_512.json → lora_chara_hibiki_v3_80i8r-82i7r.json}

@@ -3,21 +3,22 @@
   "v2": false,
   "v_parameterization": false,
   "logging_dir": "",
-  "train_data_dir": "G:/sd/training/datasets/hibiki",
-  "reg_data_dir": "G:/sd/training/datasets/regempty",
-  "output_dir": "G:/sd/repo/extensions/sd-webui-additional-networks/models/lora",
-  "max_resolution": "512,512",
+  "train_data_dir": "G:/sd/training/datasets/hibiki/dataset",
+  "reg_data_dir": "",
+  "output_dir": "G:/sd/lora/trained/chara/hibiki",
+  "max_resolution": "832,832",
+  "learning_rate": "1e-5",
   "lr_scheduler": "constant_with_warmup",
   "lr_warmup": "5",
   "train_batch_size": 3,
-  "epoch": "1",
-  "save_every_n_epochs": "1",
+  "epoch": "4",
+  "save_every_n_epochs": "",
   "mixed_precision": "fp16",
   "save_precision": "fp16",
-  "seed": "23",
+  "seed": "31337",
   "num_cpu_threads_per_process": 32,
-  "cache_latent": true,
-  "caption_extention": ".txt",
+  "cache_latents": true,
+  "caption_extension": ".txt",
   "enable_bucket": true,
   "gradient_checkpointing": false,
   "full_fp16": false,
@@ -30,8 +31,8 @@
   "save_state": false,
   "resume": "",
   "prior_loss_weight": 1.0,
-  "text_encoder_lr": "3e-5",
-  "unet_lr": "3e-4",
+  "text_encoder_lr": "1.5e-5",
+  "unet_lr": "1.5e-4",
   "network_dim": 128,
   "lora_network_weights": "",
   "color_aug": false,
@@ -39,5 +40,20 @@
   "clip_skip": 2,
   "gradient_accumulation_steps": 1.0,
   "mem_eff_attn": false,
-  "output_name": "hibiki-NAI-VAE-512px-357steps"
+  "output_name": "chara-hibiki-v3",
+  "model_list": "",
+  "max_token_length": "150",
+  "max_train_epochs": "",
+  "max_data_loader_n_workers": "",
+  "network_alpha": 128,
+  "training_comment": "Character: `hibiki, halo, dog ears, bright pupils, goggles on head, black hair, dog tail`\nStandard outfit: `twintails, hair bobbles, engineering goggles, purple choker, dog tags, multicolored jacket, black camisole, grey shirt, black shorts`\nCheerleader: `cheerleader, ponytail, crop top, wing collar, criss-cross halter, pleated miniskirt, pom pom (cheerleading)`\n\nCheerleader stickers are tagged with `sticker on face, sticker on arm, star sticker`.\nUse `engineering goggles` for her normal goggles and `blue goggles` for her cheerleader goggles.\nSeveral alternate outfits/cosplays are also included.  Use `alternate outfit` to encourage different outfits.\n\n(82 cheerleader * 7 repeats + 80 normal * 8 repeats) / 3 batch size * 4 epochs = 1772 steps",
+  "keep_tokens": 2,
+  "lr_scheduler_num_cycles": "",
+  "lr_scheduler_power": "",
+  "persistent_data_loader_workers": true,
+  "bucket_no_upscale": true,
+  "random_crop": false,
+  "bucket_reso_steps": 64.0,
+  "caption_dropout_every_n_epochs": 0.0,
+  "caption_dropout_rate": 0
 }

hibiki/lora_character_hibiki_119img9repeat_768.json

@@ -1,43 +0,0 @@
-{
-  "pretrained_model_name_or_path": "G:/sd/repo/models/Stable-diffusion/nai-animefull-final-pruned.safetensors",
-  "v2": false,
-  "v_parameterization": false,
-  "logging_dir": "",
-  "train_data_dir": "G:/sd/training/datasets/hibiki",
-  "reg_data_dir": "G:/sd/training/datasets/regempty",
-  "output_dir": "G:/sd/repo/extensions/sd-webui-additional-networks/models/lora",
-  "max_resolution": "768,768",
-  "lr_scheduler": "constant_with_warmup",
-  "lr_warmup": "5",
-  "train_batch_size": 3,
-  "epoch": "1",
-  "save_every_n_epochs": "1",
-  "mixed_precision": "fp16",
-  "save_precision": "fp16",
-  "seed": "23",
-  "num_cpu_threads_per_process": 32,
-  "cache_latent": true,
-  "caption_extention": ".txt",
-  "enable_bucket": true,
-  "gradient_checkpointing": false,
-  "full_fp16": false,
-  "no_token_padding": false,
-  "stop_text_encoder_training": 0,
-  "use_8bit_adam": true,
-  "xformers": true,
-  "save_model_as": "safetensors",
-  "shuffle_caption": true,
-  "save_state": false,
-  "resume": "",
-  "prior_loss_weight": 1.0,
-  "text_encoder_lr": "3e-5",
-  "unet_lr": "3e-4",
-  "network_dim": 128,
-  "lora_network_weights": "",
-  "color_aug": false,
-  "flip_aug": false,
-  "clip_skip": 2,
-  "gradient_accumulation_steps": 1.0,
-  "mem_eff_attn": false,
-  "output_name": "hibiki-NAI-VAE-768px-357steps"
-}

hibiki/lora_character_hibiki_split_65i6r-54i10r_768.json

@@ -1,43 +0,0 @@
-{
-  "pretrained_model_name_or_path": "G:/sd/repo/models/Stable-diffusion/nai-animefull-final-pruned.safetensors",
-  "v2": false,
-  "v_parameterization": false,
-  "logging_dir": "",
-  "train_data_dir": "G:/sd/training/datasets/hibiki",
-  "reg_data_dir": "G:/sd/training/datasets/regempty",
-  "output_dir": "G:/sd/repo/extensions/sd-webui-additional-networks/models/lora",
-  "max_resolution": "768,768",
-  "lr_scheduler": "constant_with_warmup",
-  "lr_warmup": "5",
-  "train_batch_size": 3,
-  "epoch": "1",
-  "save_every_n_epochs": "1",
-  "mixed_precision": "fp16",
-  "save_precision": "fp16",
-  "seed": "23",
-  "num_cpu_threads_per_process": 32,
-  "cache_latent": true,
-  "caption_extention": ".txt",
-  "enable_bucket": true,
-  "gradient_checkpointing": false,
-  "full_fp16": false,
-  "no_token_padding": false,
-  "stop_text_encoder_training": 0,
-  "use_8bit_adam": true,
-  "xformers": true,
-  "save_model_as": "safetensors",
-  "shuffle_caption": true,
-  "save_state": false,
-  "resume": "",
-  "prior_loss_weight": 1.0,
-  "text_encoder_lr": "3e-5",
-  "unet_lr": "3e-4",
-  "network_dim": 128,
-  "lora_network_weights": "",
-  "color_aug": false,
-  "flip_aug": false,
-  "clip_skip": 2,
-  "gradient_accumulation_steps": 1.0,
-  "mem_eff_attn": false,
-  "output_name": "hibiki-v2-NAI-VAE-768px-6.10split-310steps"
-}

hibiki/lora_character_hibiki_split_65i6r-54i10r_768_batch4_slower3epoch.json

@@ -1,43 +0,0 @@
-{
-  "pretrained_model_name_or_path": "G:/sd/repo/models/Stable-diffusion/nai-animefull-final-pruned.safetensors",
-  "v2": false,
-  "v_parameterization": false,
-  "logging_dir": "",
-  "train_data_dir": "G:/sd/training/datasets/hibiki",
-  "reg_data_dir": "G:/sd/training/datasets/regempty",
-  "output_dir": "G:/sd/repo/extensions/sd-webui-additional-networks/models/lora",
-  "max_resolution": "768,768",
-  "lr_scheduler": "constant_with_warmup",
-  "lr_warmup": "5",
-  "train_batch_size": 4,
-  "epoch": "3",
-  "save_every_n_epochs": "1",
-  "mixed_precision": "fp16",
-  "save_precision": "fp16",
-  "seed": "23",
-  "num_cpu_threads_per_process": 32,
-  "cache_latent": true,
-  "caption_extention": ".txt",
-  "enable_bucket": true,
-  "gradient_checkpointing": false,
-  "full_fp16": false,
-  "no_token_padding": false,
-  "stop_text_encoder_training": 0,
-  "use_8bit_adam": true,
-  "xformers": true,
-  "save_model_as": "safetensors",
-  "shuffle_caption": true,
-  "save_state": false,
-  "resume": "",
-  "prior_loss_weight": 1.0,
-  "text_encoder_lr": "1e-5",
-  "unet_lr": "1e-4",
-  "network_dim": 128,
-  "lora_network_weights": "",
-  "color_aug": false,
-  "flip_aug": false,
-  "clip_skip": 2,
-  "gradient_accumulation_steps": 1.0,
-  "mem_eff_attn": false,
-  "output_name": "hibiki-v3-NAI-VAE-768px-6.10split-310steps-batch4-slower"
-}

hibiki/lora_character_hibiki_split_65i6r-54i10r_768_batch5_slower4epoch.json

@@ -1,43 +0,0 @@
-{
-  "pretrained_model_name_or_path": "G:/sd/repo/models/Stable-diffusion/nai-animefull-final-pruned.safetensors",
-  "v2": false,
-  "v_parameterization": false,
-  "logging_dir": "",
-  "train_data_dir": "G:/sd/training/datasets/hibiki",
-  "reg_data_dir": "G:/sd/training/datasets/regempty",
-  "output_dir": "G:/sd/repo/extensions/sd-webui-additional-networks/models/lora",
-  "max_resolution": "768,768",
-  "lr_scheduler": "constant_with_warmup",
-  "lr_warmup": "5",
-  "train_batch_size": 5,
-  "epoch": "4",
-  "save_every_n_epochs": "1",
-  "mixed_precision": "fp16",
-  "save_precision": "fp16",
-  "seed": "23",
-  "num_cpu_threads_per_process": 32,
-  "cache_latent": true,
-  "caption_extention": ".txt",
-  "enable_bucket": true,
-  "gradient_checkpointing": false,
-  "full_fp16": false,
-  "no_token_padding": false,
-  "stop_text_encoder_training": 0,
-  "use_8bit_adam": true,
-  "xformers": true,
-  "save_model_as": "safetensors",
-  "shuffle_caption": true,
-  "save_state": false,
-  "resume": "",
-  "prior_loss_weight": 1.0,
-  "text_encoder_lr": "1e-5",
-  "unet_lr": "1e-4",
-  "network_dim": 128,
-  "lora_network_weights": "",
-  "color_aug": false,
-  "flip_aug": false,
-  "clip_skip": 2,
-  "gradient_accumulation_steps": 1.0,
-  "mem_eff_attn": false,
-  "output_name": "hibiki-v3-NAI-VAE-768px-6.10split-310steps-batch5-slower-4epoch"
-}