REILX commited on
Commit
847d8be
1 Parent(s): e480f90

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +46 -3
README.md CHANGED
@@ -1,3 +1,46 @@
1
- ---
2
- license: llama3
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: llama3
3
+ datasets:
4
+ - REILX/extracted_tagengo_gpt4
5
+ - TigerResearch/sft_zh
6
+ - alexl83/AlpacaDataCleaned
7
+ - LooksJuicy/ruozhiba
8
+ - silk-road/alpaca-data-gpt4-chinese
9
+ - databricks/databricks-dolly-15k
10
+ - microsoft/orca-math-word-problems-200k
11
+ - Sao10K/Claude-3-Opus-Instruct-5K
12
+ language:
13
+ - zh
14
+ - en
15
+ ---
16
+
17
+ ### 数据集
18
+ 使用以下8个数据集
19
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/636f54b95d2050767e4a6317/OkuVQ1lWXRAKyel2Ef0Fz.png)
20
+ 对Llama-3-8B-Instruct进行微调并测试,结果显示,微调后的模型在CEVAL和MMLU的评分上均有所提升。
21
+
22
+ ### 基础模型:
23
+ - https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct
24
+
25
+ ### 训练工具
26
+ https://github.com/hiyouga/LLaMA-Factory
27
+
28
+ ### 测评方式:
29
+ 使用opencompass(https://github.com/open-compass/OpenCompass/ ), 测试工具基于CEval和MMLU对微调之后的模型和原始模型进行测试。</br>
30
+ 测试模型分别为:
31
+ - Llama-3-8B
32
+ - Llama-3-8B-Instruct
33
+ - Llama-3-8B-Instruct-750Mb-lora, 使用8DataSets数据集对Llama-3-8B-Instruct模型进行sft方式lora微调
34
+
35
+ ### 测试机器
36
+ 8*A800
37
+
38
+ ### 8DataSets数据集:
39
+ 大约750Mb的微调数据集
40
+ - https://huggingface.co/datasets/REILX/extracted_tagengo_gpt4
41
+ - https://huggingface.co/datasets/TigerResearch/sft_zh
42
+ - https://huggingface.co/datasets/silk-road/alpaca-data-gpt4-chinese
43
+ - https://huggingface.co/datasets/LooksJuicy/ruozhiba
44
+ - https://huggingface.co/datasets/microsoft/orca-math-word-problems-200k
45
+ - https://huggingface.co/datasets/alexl83/AlpacaDataCleaned
46
+ - https://huggingface.co/datasets/Sao10K/Claude-3-Opus-Instruct-5K