Commit
•
9d5c8e9
1
Parent(s):
68987ac
Update README.md
Browse files
README.md
CHANGED
|
@@ -28,7 +28,15 @@ language:
|
|
| 28 |
- sv
|
| 29 |
---
|
| 30 |
|
| 31 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
Install packages
|
| 34 |
```Python
|
|
|
|
| 28 |
- sv
|
| 29 |
---
|
| 30 |
|
| 31 |
+
```
|
| 32 |
+
- model fine tune base: cognitivecomputations/dolphin-2_6-phi-2
|
| 33 |
+
- sft
|
| 34 |
+
- flash-attention 2
|
| 35 |
+
- loss: 0.85
|
| 36 |
+
- steps: 3000
|
| 37 |
+
- max_length: 2028
|
| 38 |
+
- neftune_noise_alpha: 5
|
| 39 |
+
```
|
| 40 |
|
| 41 |
Install packages
|
| 42 |
```Python
|