marianbasti commited on
Commit
1f96ecd
1 Parent(s): d97af6b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -0
README.md CHANGED
@@ -1,3 +1,20 @@
1
  ---
2
  license: llama2
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: llama2
3
+ datasets:
4
+ - bertin-project/alpaca-spanish
5
+ language:
6
+ - es
7
+ library_name: transformers
8
  ---
9
+ ## Llama 2-13b-alpaca-spanish LoRA
10
+ This is a LoRA for Llama 2 13B trained on a translated [alpaca dataset](https://huggingface.co/datasets/bertin-project/alpaca-spanish) on an attempt to improve spanish performance of the Llama-2 foundation model with a conversational focus.
11
+
12
+ Base model used was [The Bloke's Llama-2-13B-fp16](https://huggingface.co/TheBloke/Llama-2-13B-fp16) trained in 4bit precision with an added padding token.
13
+
14
+ | Training parameteres | |
15
+ | ----------- | ----------- |
16
+ | LoRA scale | 2 |
17
+ | Epochs | 0.75 |
18
+ | Learning Rate| 2e-5 |
19
+ | Warmup Steps| 100 |
20
+ | Loss | 1.07 |