adds retrained Mari LoRA

README.md

@@ -31,7 +31,9 @@ Here you will find the various LoRAs I've trained, typically of Blue Archive cha
 [Available on old Mega.co.nz repository.](https://mega.nz/folder/SqYwQTRI#GN2SmGTBsV6S4q-L-V4VeA)
 
 ### Mari
-[Available on old Mega.co.nz repository.](https://mega.nz/folder/SqYwQTRI#GN2SmGTBsV6S4q-L-V4VeA)
+Iochi Mari / 伊落マリー / 이오치 마리
+
+[![Mari](mari/chara-mari-v5b.png)](https://huggingface.co/khanon/lora-training/blob/main/mari/README.md)
 
 ### Michiru
 [Available on old Mega.co.nz repository.](https://mega.nz/folder/SqYwQTRI#GN2SmGTBsV6S4q-L-V4VeA)

mari/README.md

@@ -1,50 +1,65 @@
 # Iochi Mari (Blue Archive)
-I used a bigger dataset than normal since Mari has a lot of good art (253 images).
+伊落マリー (ブルーアーカイブ) / 이오치 마리 (블루 아카이브)
 
-This was a tricky LoRA to train since she has two very distinct outfits (and her gym outfit has as much art without her jacket as with). The LoRA recalls both outfits, and even her swimsuit, but you may need to negative prompt elements of whichever outfit you don't want.
+[**Download here.**](chara-mari-v5b.safetensors)
 
-## Usage
-Use any or all of these tags to summon Mari:
-`mari, halo, 1girl, blue eyes`
-`cat ears` was only tagged on images where her cowl was not present, in which case `animal ear headwear` is the tag you want.
-Hair and eyes are optional.
-
-For her normal outfit (you don't usually need all of these):
-`single braid, animal ear headwear, nun, habit, puffy long sleeves, blue neckerchief, white sailor collar`
-Her shoes were tagged `mary janes`.
-The sleeves of her outfit leak into other outfits/nudity so you may need to negative prompt them (`puffy long sleeves`)
+## Table of Contents
+- [Preview](#preview)
+- [Usage](#usage)
+- [Training](#training)
+- [Revisions](#revisions)
+- [Alternate Version](#alternate-version---v4)
 
-For her gym uniform outfit (you don't usually need all of these):
-`low ponytail, sportswear, track jacket, shorts, gym shirt, ponytail, id card, bottle`
-Images with her jacket visible were tagged `track jacket`. If the jacket was open, I added `open jacket`.
-Images with her gym shirt visible (so not those where her jacket was closed) were tagged `gym shirt`.  Adding `white` can help.
-`wet, wet clothes` was common in the trianing data and works well.
+## Preview
+![Mari portrait](chara-mari-v5b.png)
+![Mari nun](example-001-Treebark-v5b.png)
+![Mari gym](example-002-AOM2-v5b.png)
+![Mari dress](example-003-DefmixRed-v5b.png)
 
-For her swimsuit (only a handful of images in the training data, so use emphasis):
-`black one-piece swimsuit, frilled swimsuit, frills, paw print`
-Negative prompt: `competition swimsuit`
+## Usage
+Use any or all of the following tags to summon Mari: `mari, halo, 1girl, blue eyes, orange hair`
+- Hair and eyes are optional.
+- It is recommended to add `cat ears` or `animal ear headwear`, otherwise her ears may not appear consistently.  This is likely because Mari's veil prevents the AI from strongly associating her ears the word `mari`.
+- If Mari's ears are appearing through her veil, try adding `animal ear fluff` to the negative prompt.
 
-Included three epochs.  I did way more steps than normal in pursuit of perfect halo gens, 
-- epoch 7 (3542 steps) = nails her expressions and outfits pretty well but outfit details can leak over. best halo but still not great
-- epoch 6 (~3000 steps) = pretty good balance between the others
-- epoch 4 (~2000 steps) = less overfit on outfits but halo and expression consistency is not as good
+For her normal outfit: `single braid, animal ear headwear, nun, habit, blue neckerchief, white sailor collar, puffy long sleeves`
 
-Weight 1 works fine for the most part.  Epoch 7 may be somewhat stubborn/overfit on some parts of her outfit, in which case you may have better luck dropping down to epoch 6 or 4.
+For her sportswear alt: `low ponytail, sportswear, track jacket, shorts, gym shirt, ponytail, white headband, id card, bottle`
+- Use `track jacket` for a full jacket, or `open jacket` when the jacket should be open.
+- Use `wet, wet clothes` for transparent/wet clothing
+- Use `gym shirt` to discourage the full jacket from appearing, as it was only tagged when the jacket was unzipped.
 
+For her swimsuit: `black one-piece swimsuit, frilled swimsuit, frills, paw print`
+- You will likely need to add `competitions wimsuit` to the negative prompt.
 
 ## Training
-*All parameters are provided in the accompanying JSON files.*
-- Trained on a set of 253 images repeated 6 times, 7 epochs (253 images * 6 repeats / 3 batch size * 7 epochs = 3542 steps)
-  - Dataset included a mixture of SFW and NSFW.
-  - Same params as Koharu, but with almost double the amount of steps, since the dataset was large and I wasn't initially satisfied with halo/facial expression consistency
-- Initially tagged with WD1.4, then performed heavy pruning and editing.
-  - Pruned redundant tags and simplified outfits so that they were always tagged with the same handful of tags
-  - Made sure important traits were present and consitently described, and traits like `halo` were consistent with actual visibility
-  - Added many facial expression, camera angle, and image composition hints
-- Made a hard-coded tweak to bmaltais's LoRA GUI script.
-  - Previously increasing the `resolution` values (512 >> 768 >> 832) would not increase `max_bucket_reso`, so buckets were limited to 1024 (the default) in either dimension.
-  - Per the kohya_ss README, when increasing `resolution` to 768, it is recommended to increase `max_bucket_reso` to 1280 to keep the bucket aspect ratios the same as 512 resolution
-  - I adjusted the script to set `max_bucket_reso` based on `resolution`, using the formula:
-    - `max_bucket_reso = resolution * (1 + (512 / resolution))` which causes bucket aspect ratios to be the same as 512 for any given resolution.
-  - In practice, this only affected a very small number of very tall or very wide images so I don't think it would have made much of a difference
+*Exact parameters are provided in the accompanying JSON files.*
+- Trained on a set of 253 images
+  - Datset included a mixture of SFW/NSFW.
+  - 3 repeats
+  - 3 batch size, 7 epochs
+  - `(253*3) / 3 * 7` = 1771 steps
+  - Increased steps for stronger fit due to larger, more varied dataset.
+- 768x768 training resolution
+- `constant_with_warmup` scheduler
+- Initially tagged with WD1.4 Convnext model, then heavily edited.
+  - Pruned redundant tags and simplified outfit tags such that Mari's outfit was always described with the same words
+  - Ensured unique traits such as `halo` were consistent with actual visibility
+  - Added many tags for facial expressions, camera angles, and image composition
+- Used network_dimension 128 (same as usual) / network alpha 128 (default)
 - Trained without VAE.
+- [Training dataset available here.](https://mega.nz/folder/2uYkWQiA#GlVNAYXubskusIMbrmVOGQ)
+
+## Revisions
+- v5b (2023-02-08)
+  - Completely re-trained to correct issue where caption files were not being loaded correctly.  New version should have more flexibility but requires additional outfit prompting.
+- v4 (2023-01-20)
+  - Initial release.
+  - **Issue:** caption files were completely ignored during training. As a result, prompting "sora" grants a very strong effect, but the LoRA is highly overfit.
+
+## Alternate Version - v4
+**v4** is also available. This version was improperly configured and ignored caption files, so it was trained only on the word `mari`.  It looks quite good, however it is very overfit and it is difficult to adjust Mari's outfit.  In the example image below, the prompt includes `yellow dress` but the model ignores the color and instead uses a dress that resembles Mari's usual outfit.
+
+**[Download v4.](v4/chara-mari-v4-epoch7.safetensors)**
+
+![v4 Preview](v4/example-003-DefmixRed-v4.png)

mari/chara-mari-v5b.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8cb3f9d5f5fe82845f1ced4d7236de1bb85ba1deb513e49122e5e3dcc08cd1da
+size 151128976

mari/{lora_character_mari_v2_253i6r_768-fixed-max-bucket-reso_batch4_5e-5text_5e-4unet_4epoch.json → lora_chara_mari_v5b_253i3r.json}

@@ -3,22 +3,22 @@
   "v2": false,
   "v_parameterization": false,
   "logging_dir": "",
-  "train_data_dir": "G:/sd/training/datasets/mari",
-  "reg_data_dir": "G:/sd/training/datasets/regempty",
-  "output_dir": "G:/sd/repo/extensions/sd-webui-additional-networks/models/lora",
+  "train_data_dir": "G:/sd/training/datasets/mari/dataset",
+  "reg_data_dir": "",
+  "output_dir": "G:/sd/lora/trained/chara/mari",
   "max_resolution": "768,768",
-  "learning_rate": "",
-  "lr_scheduler": "cosine_with_restarts",
+  "learning_rate": "1e-5",
+  "lr_scheduler": "constant_with_warmup",
   "lr_warmup": "5",
-  "train_batch_size": 4,
-  "epoch": "5",
-  "save_every_n_epochs": "1",
+  "train_batch_size": 3,
+  "epoch": "7",
+  "save_every_n_epochs": "6",
   "mixed_precision": "fp16",
   "save_precision": "fp16",
   "seed": "31337",
   "num_cpu_threads_per_process": 32,
   "cache_latents": true,
-  "caption_extension": "",
+  "caption_extension": ".txt",
   "enable_bucket": true,
   "gradient_checkpointing": false,
   "full_fp16": false,
@@ -31,8 +31,8 @@
   "save_state": false,
   "resume": "",
   "prior_loss_weight": 1.0,
-  "text_encoder_lr": "5e-5",
-  "unet_lr": "5e-4",
+  "text_encoder_lr": "1.5e-5",
+  "unet_lr": "1.5e-4",
   "network_dim": 128,
   "lora_network_weights": "",
   "color_aug": false,
@@ -40,9 +40,15 @@
   "clip_skip": 2,
   "gradient_accumulation_steps": 1.0,
   "mem_eff_attn": false,
-  "output_name": "mari-v2-NoVAE",
+  "output_name": "chara-mari-v5b",
   "model_list": "",
   "max_token_length": "150",
   "max_train_epochs": "",
-  "max_data_loader_n_workers": ""
+  "max_data_loader_n_workers": "",
+  "network_alpha": 128,
+  "training_comment": "Character: `mari, halo, 1girl, orange hair, blue eyes` (hair and eyes optional)\nStandard outfit: `single braid, nun, animal ear headwear, habit, puffy long sleeves, blue neckerchief, white sailor collar`\nGym outfit: `low ponytail, sportswear, cat ears, animal ear fluff, track jacket, shorts, gym shirt, ponytail, id card, bottle`\n253 images * 3 repeats / 3 batch size * 7 epochs = 1771 steps",
+  "keep_tokens": 2,
+  "lr_scheduler_num_cycles": "",
+  "lr_scheduler_power": "",
+  "persistent_data_loader_workers": true
 }

mari/lora_character_mari_v1_253i8r_832_batch3_5e-5text_2e-4unet_4epoch.json

@@ -1,48 +0,0 @@
-{
-  "pretrained_model_name_or_path": "G:/sd/repo/models/Stable-diffusion/nai-animefull-final-pruned.safetensors",
-  "v2": false,
-  "v_parameterization": false,
-  "logging_dir": "",
-  "train_data_dir": "G:/sd/training/datasets/mari",
-  "reg_data_dir": "G:/sd/training/datasets/regempty",
-  "output_dir": "G:/sd/repo/extensions/sd-webui-additional-networks/models/lora",
-  "max_resolution": "832,832",
-  "learning_rate": "",
-  "lr_scheduler": "cosine_with_restarts",
-  "lr_warmup": "5",
-  "train_batch_size": 3,
-  "epoch": "6",
-  "save_every_n_epochs": "2",
-  "mixed_precision": "fp16",
-  "save_precision": "fp16",
-  "seed": "31337",
-  "num_cpu_threads_per_process": 32,
-  "cache_latents": true,
-  "caption_extension": "",
-  "enable_bucket": true,
-  "gradient_checkpointing": false,
-  "full_fp16": false,
-  "no_token_padding": false,
-  "stop_text_encoder_training": 0,
-  "use_8bit_adam": true,
-  "xformers": true,
-  "save_model_as": "safetensors",
-  "shuffle_caption": true,
-  "save_state": false,
-  "resume": "",
-  "prior_loss_weight": 1.0,
-  "text_encoder_lr": "5e-5",
-  "unet_lr": "2e-4",
-  "network_dim": 128,
-  "lora_network_weights": "",
-  "color_aug": false,
-  "flip_aug": false,
-  "clip_skip": 2,
-  "gradient_accumulation_steps": 1.0,
-  "mem_eff_attn": false,
-  "output_name": "mari-v3-NoVAE",
-  "model_list": "",
-  "max_token_length": "150",
-  "max_train_epochs": "",
-  "max_data_loader_n_workers": ""
-}

mari/lora_character_mari_v3_253i8r_832_batch3_5e-5text_3e-4unet_4epoch.json

@@ -1,48 +0,0 @@
-{
-  "pretrained_model_name_or_path": "G:/sd/repo/models/Stable-diffusion/nai-animefull-final-pruned.safetensors",
-  "v2": false,
-  "v_parameterization": false,
-  "logging_dir": "",
-  "train_data_dir": "G:/sd/training/datasets/mari",
-  "reg_data_dir": "G:/sd/training/datasets/regempty",
-  "output_dir": "G:/sd/repo/extensions/sd-webui-additional-networks/models/lora",
-  "max_resolution": "832,832",
-  "learning_rate": "",
-  "lr_scheduler": "cosine_with_restarts",
-  "lr_warmup": "5",
-  "train_batch_size": 3,
-  "epoch": "4",
-  "save_every_n_epochs": "3",
-  "mixed_precision": "fp16",
-  "save_precision": "fp16",
-  "seed": "23",
-  "num_cpu_threads_per_process": 32,
-  "cache_latents": true,
-  "caption_extension": "",
-  "enable_bucket": true,
-  "gradient_checkpointing": false,
-  "full_fp16": false,
-  "no_token_padding": false,
-  "stop_text_encoder_training": 0,
-  "use_8bit_adam": true,
-  "xformers": true,
-  "save_model_as": "safetensors",
-  "shuffle_caption": true,
-  "save_state": false,
-  "resume": "",
-  "prior_loss_weight": 1.0,
-  "text_encoder_lr": "5e-5",
-  "unet_lr": "3e-4",
-  "network_dim": 128,
-  "lora_network_weights": "",
-  "color_aug": false,
-  "flip_aug": false,
-  "clip_skip": 2,
-  "gradient_accumulation_steps": 1.0,
-  "mem_eff_attn": false,
-  "output_name": "mari-v3-NoVAE",
-  "model_list": "",
-  "max_token_length": "150",
-  "max_train_epochs": "",
-  "max_data_loader_n_workers": ""
-}

mari/lora_character_mari_v4_253i6r_832_batch3_5e-5text_2e-4unet_6epoch.json

@@ -1,48 +0,0 @@
-{
-  "pretrained_model_name_or_path": "G:/sd/repo/models/Stable-diffusion/nai-animefull-final-pruned.safetensors",
-  "v2": false,
-  "v_parameterization": false,
-  "logging_dir": "",
-  "train_data_dir": "G:/sd/training/datasets/mari",
-  "reg_data_dir": "G:/sd/training/datasets/regempty",
-  "output_dir": "G:/sd/repo/extensions/sd-webui-additional-networks/models/lora",
-  "max_resolution": "832,832",
-  "learning_rate": "",
-  "lr_scheduler": "cosine_with_restarts",
-  "lr_warmup": "5",
-  "train_batch_size": 3,
-  "epoch": "6",
-  "save_every_n_epochs": "2",
-  "mixed_precision": "fp16",
-  "save_precision": "fp16",
-  "seed": "31337",
-  "num_cpu_threads_per_process": 32,
-  "cache_latents": true,
-  "caption_extension": "",
-  "enable_bucket": true,
-  "gradient_checkpointing": false,
-  "full_fp16": false,
-  "no_token_padding": false,
-  "stop_text_encoder_training": 0,
-  "use_8bit_adam": true,
-  "xformers": true,
-  "save_model_as": "safetensors",
-  "shuffle_caption": true,
-  "save_state": false,
-  "resume": "",
-  "prior_loss_weight": 1.0,
-  "text_encoder_lr": "5e-5",
-  "unet_lr": "2e-4",
-  "network_dim": 128,
-  "lora_network_weights": "",
-  "color_aug": false,
-  "flip_aug": false,
-  "clip_skip": 2,
-  "gradient_accumulation_steps": 1.0,
-  "mem_eff_attn": false,
-  "output_name": "mari-v4-NoVAE",
-  "model_list": "",
-  "max_token_length": "150",
-  "max_train_epochs": "",
-  "max_data_loader_n_workers": ""
-}

mari/v4/chara-mari-v4-epoch7.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2fa63a5b21117b75f8a3b92f274392e13f21568e11d31eb0c75bdcc3216afabc
+size 151074586