UsernameJustAnother commited on
Commit
095a44a
1 Parent(s): 85f843f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -1
README.md CHANGED
@@ -20,7 +20,14 @@ tags:
20
  Experimental RP Finetune with secret sauce dataset, rsLoRA, r = 64, on an Colab A100 instance. 30GB vRAM used, 2 epochs ~ 3hrs of training.
21
 
22
  ```
23
- r = 64,
 
 
 
 
 
 
 
24
  target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
25
  "gate_proj", "up_proj", "down_proj",],
26
  lora_alpha = 64,
 
20
  Experimental RP Finetune with secret sauce dataset, rsLoRA, r = 64, on an Colab A100 instance. 30GB vRAM used, 2 epochs ~ 3hrs of training.
21
 
22
  ```
23
+ ==((====))== Unsloth - 2x faster free finetuning | Num GPUs = 1
24
+ \\ /| Num examples = 8,160 | Num Epochs = 2
25
+ O^O/ \_/ \ Batch size per device = 2 | Gradient Accumulation steps = 4
26
+ \ / Total batch size = 8 | Total steps = 2,040
27
+ "-____-" Number of trainable parameters = 228,065,280
28
+
29
+
30
+ r = 64,
31
  target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
32
  "gate_proj", "up_proj", "down_proj",],
33
  lora_alpha = 64,