chiliu commited on
Commit
e9e5249
1 Parent(s): 21a8212

update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -0
README.md CHANGED
@@ -12,6 +12,19 @@ thumbnail: >-
12
  license: apache-2.0
13
  ---
14
  # Model Card
 
 
 
 
 
 
 
 
 
 
 
 
 
15
  ## Summary
16
 
17
  We have fine-tuned the open-lama model and surpassed the original model in multiple evaluation subtasks, making it currently the best performing 3B model with comparable performance to llama-7b
 
12
  license: apache-2.0
13
  ---
14
  # Model Card
15
+ ## Github
16
+ https://github.com/chi2liu/mamba-gpt-3b
17
+
18
+ | Metric | Value |
19
+ |-----------------------|-------|
20
+ | MMLU (5-shot) | 25.3 |
21
+ | ARC (25-shot) | 40.5 |
22
+ | HellaSwag (10-shot) | 64.9 |
23
+ | TruthfulQA (0-shot) | 37.1 |
24
+ | Avg. | 42.0 |
25
+
26
+ We use state-of-the-art [Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) to run the benchmark tests above.
27
+
28
  ## Summary
29
 
30
  We have fine-tuned the open-lama model and surpassed the original model in multiple evaluation subtasks, making it currently the best performing 3B model with comparable performance to llama-7b