Update README.md
Browse files
README.md
CHANGED
@@ -30,6 +30,14 @@ This modelcard aims to be a base template for new models. It has been generated
|
|
30 |
|
31 |
<!-- Provide a longer summary of what this model is. -->
|
32 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
33 |
|
34 |
|
35 |
- **Developed by:** lee12ki
|
|
|
30 |
|
31 |
<!-- Provide a longer summary of what this model is. -->
|
32 |
|
33 |
+
This model, lee12ki/llama2-finetune-7b, is a fine-tuned version of NousResearch/Llama-2-7b-chat-hf. It has been optimized for text-generation tasks, particularly for instruction-following and conversational applications. The fine-tuning was performed using the Guanaco-LLaMA2-1K dataset, a high-quality dataset designed for aligning language models to human preferences.
|
34 |
+
|
35 |
+
The fine-tuning process utilizes QLoRA (Quantized Low-Rank Adaptation), which enables efficient and memory-friendly training with 4-bit precision. LoRA configuration parameters include:
|
36 |
+
|
37 |
+
LoRA rank (r): 64
|
38 |
+
Alpha parameter: 16
|
39 |
+
Dropout probability: 0.1
|
40 |
+
|
41 |
|
42 |
|
43 |
- **Developed by:** lee12ki
|