Adding Evaluation Results

README.md

@@ -1,15 +1,13 @@
 ---
 language:
 - en
-library_name: transformers
 license: apache-2.0
+library_name: transformers
 tags:
 - gpt
 - llm
 - large language model
 - h2o-llmstudio
-thumbnail: >-
-  https://h2o.ai/etc.clientlibs/h2o/clientlibs/clientlib-site/resources/images/favicon.ico
 datasets:
 - HuggingFaceH4/ultrafeedback_binarized
 - Intel/orca_dpo_pairs
@@ -18,8 +16,112 @@ datasets:
 - OpenAssistant/oasst2
 - HuggingFaceH4/ultrachat_200k
 - meta-math/MetaMathQA
+thumbnail: https://h2o.ai/etc.clientlibs/h2o/clientlibs/clientlib-site/resources/images/favicon.ico
 widget:
-- text: "<|prompt|>Why is drinking water so healthy?</s><|answer|>"
+- text: <|prompt|>Why is drinking water so healthy?</s><|answer|>
+model-index:
+- name: h2o-danube-1.8b-chat
+  results:
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: AI2 Reasoning Challenge (25-Shot)
+      type: ai2_arc
+      config: ARC-Challenge
+      split: test
+      args:
+        num_few_shot: 25
+    metrics:
+    - type: acc_norm
+      value: 41.13
+      name: normalized accuracy
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=h2oai/h2o-danube-1.8b-chat
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: HellaSwag (10-Shot)
+      type: hellaswag
+      split: validation
+      args:
+        num_few_shot: 10
+    metrics:
+    - type: acc_norm
+      value: 68.06
+      name: normalized accuracy
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=h2oai/h2o-danube-1.8b-chat
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: MMLU (5-Shot)
+      type: cais/mmlu
+      config: all
+      split: test
+      args:
+        num_few_shot: 5
+    metrics:
+    - type: acc
+      value: 33.41
+      name: accuracy
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=h2oai/h2o-danube-1.8b-chat
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: TruthfulQA (0-shot)
+      type: truthful_qa
+      config: multiple_choice
+      split: validation
+      args:
+        num_few_shot: 0
+    metrics:
+    - type: mc2
+      value: 41.64
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=h2oai/h2o-danube-1.8b-chat
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: Winogrande (5-shot)
+      type: winogrande
+      config: winogrande_xl
+      split: validation
+      args:
+        num_few_shot: 5
+    metrics:
+    - type: acc
+      value: 65.35
+      name: accuracy
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=h2oai/h2o-danube-1.8b-chat
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: GSM8k (5-shot)
+      type: gsm8k
+      config: main
+      split: test
+      args:
+        num_few_shot: 5
+    metrics:
+    - type: acc
+      value: 17.36
+      name: accuracy
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=h2oai/h2o-danube-1.8b-chat
+      name: Open LLM Leaderboard
 ---
 # Model Card
 ## Summary
@@ -152,4 +254,17 @@ Please read this disclaimer carefully before using the large language model prov
 - Reporting Issues: If you encounter any biased, offensive, or otherwise inappropriate content generated by the large language model, please report it to the repository maintainers through the provided channels. Your feedback will help improve the model and mitigate potential issues.
 - Changes to this Disclaimer: The developers of this repository reserve the right to modify or update this disclaimer at any time without prior notice. It is the user's responsibility to periodically review the disclaimer to stay informed about any changes.
 
-By using the large language model provided in this repository, you agree to accept and comply with the terms and conditions outlined in this disclaimer. If you do not agree with any part of this disclaimer, you should refrain from using the model and any content generated by it.
+By using the large language model provided in this repository, you agree to accept and comply with the terms and conditions outlined in this disclaimer. If you do not agree with any part of this disclaimer, you should refrain from using the model and any content generated by it.
+# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
+Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_h2oai__h2o-danube-1.8b-chat)
+
+|             Metric              |Value|
+|---------------------------------|----:|
+|Avg.                             |44.49|
+|AI2 Reasoning Challenge (25-Shot)|41.13|
+|HellaSwag (10-Shot)              |68.06|
+|MMLU (5-Shot)                    |33.41|
+|TruthfulQA (0-shot)              |41.64|
+|Winogrande (5-shot)              |65.35|
+|GSM8k (5-shot)                   |17.36|
+