README.md

metadata

base_model: jsfs11/MixtureofMerges-MoE-2x7b-SLERPv0.9
inference: false
library_name: transformers
license: apache-2.0
merged_models:
  - jsfs11/MixtureofMerges-MoE-2x7b-v7
  - jsfs11/MixtureofMerges-MoE-2x7bRP-v8
model-index:
  - name: MixtureofMerges-MoE-2x7b-SLERPv0.9
    results:
      - dataset:
          args:
            num_few_shot: 25
          config: ARC-Challenge
          name: AI2 Reasoning Challenge (25-Shot)
          split: test
          type: ai2_arc
        metrics:
          - name: normalized accuracy
            type: acc_norm
            value: 73.12
        source:
          name: Open LLM Leaderboard
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=jsfs11/MixtureofMerges-MoE-2x7b-SLERPv0.9
        task:
          name: Text Generation
          type: text-generation
      - dataset:
          args:
            num_few_shot: 10
          name: HellaSwag (10-Shot)
          split: validation
          type: hellaswag
        metrics:
          - name: normalized accuracy
            type: acc_norm
            value: 88.76
        source:
          name: Open LLM Leaderboard
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=jsfs11/MixtureofMerges-MoE-2x7b-SLERPv0.9
        task:
          name: Text Generation
          type: text-generation
      - dataset:
          args:
            num_few_shot: 5
          config: all
          name: MMLU (5-Shot)
          split: test
          type: cais/mmlu
        metrics:
          - name: accuracy
            type: acc
            value: 65
        source:
          name: Open LLM Leaderboard
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=jsfs11/MixtureofMerges-MoE-2x7b-SLERPv0.9
        task:
          name: Text Generation
          type: text-generation
      - dataset:
          args:
            num_few_shot: 0
          config: multiple_choice
          name: TruthfulQA (0-shot)
          split: validation
          type: truthful_qa
        metrics:
          - type: mc2
            value: 74.83
        source:
          name: Open LLM Leaderboard
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=jsfs11/MixtureofMerges-MoE-2x7b-SLERPv0.9
        task:
          name: Text Generation
          type: text-generation
      - dataset:
          args:
            num_few_shot: 5
          config: winogrande_xl
          name: Winogrande (5-shot)
          split: validation
          type: winogrande
        metrics:
          - name: accuracy
            type: acc
            value: 83.58
        source:
          name: Open LLM Leaderboard
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=jsfs11/MixtureofMerges-MoE-2x7b-SLERPv0.9
        task:
          name: Text Generation
          type: text-generation
      - dataset:
          args:
            num_few_shot: 5
          config: main
          name: GSM8k (5-shot)
          split: test
          type: gsm8k
        metrics:
          - name: accuracy
            type: acc
            value: 69.22
        source:
          name: Open LLM Leaderboard
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=jsfs11/MixtureofMerges-MoE-2x7b-SLERPv0.9
        task:
          name: Text Generation
          type: text-generation
pipeline_tag: text-generation
quantized_by: Suparious
tags:
  - 4-bit
  - AWQ
  - text-generation
  - autotrain_compatible
  - endpoints_compatible
  - merge
  - mergekit
  - lazymergekit
  - jsfs11/MixtureofMerges-MoE-2x7b-v7
  - jsfs11/MixtureofMerges-MoE-2x7bRP-v8

jsfs11/MixtureofMerges-MoE-2x7b-SLERPv0.9 AWQ

• Model creator: jsfs11
• Original model: MixtureofMerges-MoE-2x7b-SLERPv0.9

Model Summary

MixtureofMerges-MoE-2x7b-SLERPv0.9 is a merge of the following models using LazyMergekit:

• jsfs11/MixtureofMerges-MoE-2x7b-v7
• jsfs11/MixtureofMerges-MoE-2x7bRP-v8