Update README.md
Browse files
README.md
CHANGED
|
@@ -69,6 +69,32 @@ GPT-4All Benchmark Set
|
|
| 69 |
Average: 0.679
|
| 70 |
```
|
| 71 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 72 |
TruthfulQA:
|
| 73 |
```
|
| 74 |
hf-causal-experimental (pretrained=teknium/OpenHermes-7B,dtype=float16), limit: None, provide_description: False, num_fewshot: 0, batch_size: 8
|
|
|
|
| 69 |
Average: 0.679
|
| 70 |
```
|
| 71 |
|
| 72 |
+
BigBench:
|
| 73 |
+
```
|
| 74 |
+
| Task |Version| Metric |Value | |Stderr|
|
| 75 |
+
|------------------------------------------------|------:|---------------------|-----:|---|-----:|
|
| 76 |
+
|bigbench_causal_judgement | 0|multiple_choice_grade|0.5000|± |0.0364|
|
| 77 |
+
|bigbench_date_understanding | 0|multiple_choice_grade|0.5908|± |0.0256|
|
| 78 |
+
|bigbench_disambiguation_qa | 0|multiple_choice_grade|0.3023|± |0.0286|
|
| 79 |
+
|bigbench_geometric_shapes | 0|multiple_choice_grade|0.1003|± |0.0159|
|
| 80 |
+
| | |exact_str_match |0.0000|± |0.0000|
|
| 81 |
+
|bigbench_logical_deduction_five_objects | 0|multiple_choice_grade|0.2520|± |0.0194|
|
| 82 |
+
|bigbench_logical_deduction_seven_objects | 0|multiple_choice_grade|0.1871|± |0.0148|
|
| 83 |
+
|bigbench_logical_deduction_three_objects | 0|multiple_choice_grade|0.3833|± |0.0281|
|
| 84 |
+
|bigbench_movie_recommendation | 0|multiple_choice_grade|0.2500|± |0.0194|
|
| 85 |
+
|bigbench_navigate | 0|multiple_choice_grade|0.5000|± |0.0158|
|
| 86 |
+
|bigbench_reasoning_about_colored_objects | 0|multiple_choice_grade|0.4370|± |0.0111|
|
| 87 |
+
|bigbench_ruin_names | 0|multiple_choice_grade|0.2679|± |0.0209|
|
| 88 |
+
|bigbench_salient_translation_error_detection | 0|multiple_choice_grade|0.2495|± |0.0137|
|
| 89 |
+
|bigbench_snarks | 0|multiple_choice_grade|0.5249|± |0.0372|
|
| 90 |
+
|bigbench_sports_understanding | 0|multiple_choice_grade|0.5406|± |0.0159|
|
| 91 |
+
|bigbench_temporal_sequences | 0|multiple_choice_grade|0.2470|± |0.0136|
|
| 92 |
+
|bigbench_tracking_shuffled_objects_five_objects | 0|multiple_choice_grade|0.1944|± |0.0112|
|
| 93 |
+
|bigbench_tracking_shuffled_objects_seven_objects| 0|multiple_choice_grade|0.1509|± |0.0086|
|
| 94 |
+
|bigbench_tracking_shuffled_objects_three_objects| 0|multiple_choice_grade|0.3833|± |0.0281|
|
| 95 |
+
Average: 0.3367
|
| 96 |
+
```
|
| 97 |
+
|
| 98 |
TruthfulQA:
|
| 99 |
```
|
| 100 |
hf-causal-experimental (pretrained=teknium/OpenHermes-7B,dtype=float16), limit: None, provide_description: False, num_fewshot: 0, batch_size: 8
|