Adding Evaluation Results

README.md

@@ -1,37 +1,152 @@
 ---
-pipeline_tag: text-generation
-base_model: Locutusque/TinyMistral-248M
+language:
+- en
 license: apache-2.0
 datasets:
 - Locutusque/InstructMixCleaned
 - berkeley-nest/Nectar
-language:
-- en
+pipeline_tag: text-generation
+base_model: Locutusque/TinyMistral-248M
 widget:
-  - text: >-
-      <|USER|> Design a Neo4j database and Cypher function snippet to Display
-      Extreme Dental hygiene: Using Mouthwash for Analysis for Beginners.
-      Implement if/else or switch/case statements to handle different conditions
-      related to the Consent. Provide detailed comments explaining your control
-      flow and the reasoning behind each decision. <|ASSISTANT|> 
-  - text: >-
-      <|USER|> Write me a story about a magical place. <|ASSISTANT|> 
-  - text: >-
-      <|USER|> Write me an essay about the life of George Washington <|ASSISTANT|> 
-  - text: >-
-      <|USER|> Solve the following equation 2x + 10 = 20 <|ASSISTANT|> 
-  - text: >-
-      <|USER|> Craft me a list of some nice places to visit around the world. <|ASSISTANT|> 
-  - text: >-
-      <|USER|> How to manage a lazy employee: Address the employee verbally. Don't allow an employee's laziness or lack of enthusiasm to become a recurring issue. Tell the employee you're hoping to speak with them about workplace expectations and performance, and schedule a time to sit down together. Question: To manage a lazy employee, it is suggested to talk to the employee. True, False, or Neither? <|ASSISTANT|> 
+- text: '<|USER|> Design a Neo4j database and Cypher function snippet to Display Extreme
+    Dental hygiene: Using Mouthwash for Analysis for Beginners. Implement if/else
+    or switch/case statements to handle different conditions related to the Consent.
+    Provide detailed comments explaining your control flow and the reasoning behind
+    each decision. <|ASSISTANT|> '
+- text: '<|USER|> Write me a story about a magical place. <|ASSISTANT|> '
+- text: '<|USER|> Write me an essay about the life of George Washington <|ASSISTANT|> '
+- text: '<|USER|> Solve the following equation 2x + 10 = 20 <|ASSISTANT|> '
+- text: '<|USER|> Craft me a list of some nice places to visit around the world. <|ASSISTANT|> '
+- text: '<|USER|> How to manage a lazy employee: Address the employee verbally. Don''t
+    allow an employee''s laziness or lack of enthusiasm to become a recurring issue.
+    Tell the employee you''re hoping to speak with them about workplace expectations
+    and performance, and schedule a time to sit down together. Question: To manage
+    a lazy employee, it is suggested to talk to the employee. True, False, or Neither?
+    <|ASSISTANT|> '
 inference:
   parameters:
     temperature: 0.5
-    do_sample: True
+    do_sample: true
     top_p: 0.5
     top_k: 30
     max_new_tokens: 250
     repetition_penalty: 1.15
+model-index:
+- name: TinyMistral-248M-Instruct
+  results:
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: AI2 Reasoning Challenge (25-Shot)
+      type: ai2_arc
+      config: ARC-Challenge
+      split: test
+      args:
+        num_few_shot: 25
+    metrics:
+    - type: acc_norm
+      value: 24.32
+      name: normalized accuracy
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Locutusque/TinyMistral-248M-Instruct
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: HellaSwag (10-Shot)
+      type: hellaswag
+      split: validation
+      args:
+        num_few_shot: 10
+    metrics:
+    - type: acc_norm
+      value: 27.52
+      name: normalized accuracy
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Locutusque/TinyMistral-248M-Instruct
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: MMLU (5-Shot)
+      type: cais/mmlu
+      config: all
+      split: test
+      args:
+        num_few_shot: 5
+    metrics:
+    - type: acc
+      value: 25.18
+      name: accuracy
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Locutusque/TinyMistral-248M-Instruct
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: TruthfulQA (0-shot)
+      type: truthful_qa
+      config: multiple_choice
+      split: validation
+      args:
+        num_few_shot: 0
+    metrics:
+    - type: mc2
+      value: 41.94
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Locutusque/TinyMistral-248M-Instruct
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: Winogrande (5-shot)
+      type: winogrande
+      config: winogrande_xl
+      split: validation
+      args:
+        num_few_shot: 5
+    metrics:
+    - type: acc
+      value: 50.2
+      name: accuracy
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Locutusque/TinyMistral-248M-Instruct
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: GSM8k (5-shot)
+      type: gsm8k
+      config: main
+      split: test
+      args:
+        num_few_shot: 5
+    metrics:
+    - type: acc
+      value: 0.0
+      name: accuracy
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Locutusque/TinyMistral-248M-Instruct
+      name: Open LLM Leaderboard
 ---
 Base model Locutusque/TinyMistral-248M fully fine-tuned on Locutusque/InstructMix. During validation, this model achieved an average perplexity of 3.23 on Locutusque/InstructMix dataset.
-It has so far been trained on approximately 608,000 examples. More epochs are planned for this model.
+It has so far been trained on approximately 608,000 examples. More epochs are planned for this model.
+# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
+Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Locutusque__TinyMistral-248M-Instruct)
+
+|             Metric              |Value|
+|---------------------------------|----:|
+|Avg.                             |28.19|
+|AI2 Reasoning Challenge (25-Shot)|24.32|
+|HellaSwag (10-Shot)              |27.52|
+|MMLU (5-Shot)                    |25.18|
+|TruthfulQA (0-shot)              |41.94|
+|Winogrande (5-shot)              |50.20|
+|GSM8k (5-shot)                   | 0.00|
+