lee12ki commited on
Commit
3c28d6b
1 Parent(s): 59d3697

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -0
README.md CHANGED
@@ -30,6 +30,14 @@ This modelcard aims to be a base template for new models. It has been generated
30
 
31
  <!-- Provide a longer summary of what this model is. -->
32
 
 
 
 
 
 
 
 
 
33
 
34
 
35
  - **Developed by:** lee12ki
 
30
 
31
  <!-- Provide a longer summary of what this model is. -->
32
 
33
+ This model, lee12ki/llama2-finetune-7b, is a fine-tuned version of NousResearch/Llama-2-7b-chat-hf. It has been optimized for text-generation tasks, particularly for instruction-following and conversational applications. The fine-tuning was performed using the Guanaco-LLaMA2-1K dataset, a high-quality dataset designed for aligning language models to human preferences.
34
+
35
+ The fine-tuning process utilizes QLoRA (Quantized Low-Rank Adaptation), which enables efficient and memory-friendly training with 4-bit precision. LoRA configuration parameters include:
36
+
37
+ LoRA rank (r): 64
38
+ Alpha parameter: 16
39
+ Dropout probability: 0.1
40
+
41
 
42
 
43
  - **Developed by:** lee12ki