UsernameJustAnother
commited on
Commit
•
095a44a
1
Parent(s):
85f843f
Update README.md
Browse files
README.md
CHANGED
@@ -20,7 +20,14 @@ tags:
|
|
20 |
Experimental RP Finetune with secret sauce dataset, rsLoRA, r = 64, on an Colab A100 instance. 30GB vRAM used, 2 epochs ~ 3hrs of training.
|
21 |
|
22 |
```
|
23 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
24 |
target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
|
25 |
"gate_proj", "up_proj", "down_proj",],
|
26 |
lora_alpha = 64,
|
|
|
20 |
Experimental RP Finetune with secret sauce dataset, rsLoRA, r = 64, on an Colab A100 instance. 30GB vRAM used, 2 epochs ~ 3hrs of training.
|
21 |
|
22 |
```
|
23 |
+
==((====))== Unsloth - 2x faster free finetuning | Num GPUs = 1
|
24 |
+
\\ /| Num examples = 8,160 | Num Epochs = 2
|
25 |
+
O^O/ \_/ \ Batch size per device = 2 | Gradient Accumulation steps = 4
|
26 |
+
\ / Total batch size = 8 | Total steps = 2,040
|
27 |
+
"-____-" Number of trainable parameters = 228,065,280
|
28 |
+
|
29 |
+
|
30 |
+
r = 64,
|
31 |
target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
|
32 |
"gate_proj", "up_proj", "down_proj",],
|
33 |
lora_alpha = 64,
|