File size: 7,697 Bytes
---
license: apache-2.0
tags:
- moe
- frankenmoe
- merge
- mergekit
- Himitsui/Kaiju-11B
- Sao10K/Fimbulvetr-11B-v2
- decapoda-research/Antares-11b-v2
- beberik/Nyxene-v3-11B
base_model:
- Himitsui/Kaiju-11B
- Sao10K/Fimbulvetr-11B-v2
- decapoda-research/Antares-11b-v2
- beberik/Nyxene-v3-11B
model-index:
- name: Umbra-v3-MoE-4x11b
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: AI2 Reasoning Challenge (25-Shot)
      type: ai2_arc
      config: ARC-Challenge
      split: test
      args:
        num_few_shot: 25
    metrics:
    - type: acc_norm
      value: 68.43
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Steelskull/Umbra-v3-MoE-4x11b
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: HellaSwag (10-Shot)
      type: hellaswag
      split: validation
      args:
        num_few_shot: 10
    metrics:
    - type: acc_norm
      value: 87.83
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Steelskull/Umbra-v3-MoE-4x11b
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU (5-Shot)
      type: cais/mmlu
      config: all
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 65.99
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Steelskull/Umbra-v3-MoE-4x11b
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: TruthfulQA (0-shot)
      type: truthful_qa
      config: multiple_choice
      split: validation
      args:
        num_few_shot: 0
    metrics:
    - type: mc2
      value: 69.3
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Steelskull/Umbra-v3-MoE-4x11b
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: Winogrande (5-shot)
      type: winogrande
      config: winogrande_xl
      split: validation
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 83.9
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Steelskull/Umbra-v3-MoE-4x11b
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GSM8k (5-shot)
      type: gsm8k
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 63.08
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Steelskull/Umbra-v3-MoE-4x11b
      name: Open LLM Leaderboard
---
<!DOCTYPE html>
<style>
    body {
    font-family: 'Quicksand', sans-serif;
    background: linear-gradient(135deg, #2E3440 0%, #1A202C 100%);
    color: #D8DEE9;
    margin: 0;
    padding: 0;
    font-size: 16px;
}

.container {
    width: 80%;
    max-width: 800px;
    margin: 20px auto;
    background-color: rgba(255, 255, 255, 0.02);
    padding: 20px;
    border-radius: 12px;
    box-shadow: 0 4px 10px rgba(0, 0, 0, 0.2);
    backdrop-filter: blur(10px);
    border: 1px solid rgba(255, 255, 255, 0.1);
}

.header h1 {
    font-size: 28px;
    color: #ECEFF4;
    margin: 0 0 20px 0;
    text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.3);
}

.update-section {
    margin-top: 30px;
}

.update-section h2 {
    font-size: 24px;
    color: #88C0D0;
}

.update-section p {
    font-size: 16px;
    line-height: 1.6;
    color: #ECEFF4;
}

.info img {
    width: 100%;
    border-radius: 10px;
    margin-bottom: 15px;
}

a {
    color: #88C0D0;
    text-decoration: none;
}

a:hover {
    color: #A3BE8C;
}

.button {
    display: inline-block;
    background-color: #5E81AC;
    color: #E5E9F0;
    padding: 10px 20px;
    border-radius: 5px;
    cursor: pointer;
    text-decoration: none;
}

.button:hover {
    background-color: #81A1C1;
}

</style>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Umbra-v3-MoE-4x11b Data Card</title>
    <link href="https://fonts.googleapis.com/css2?family=Quicksand:wght@400;500;600&display=swap" rel="stylesheet">
</head>
<body>
<div class="container">
    <div class="header">
        <h1>Umbra-v3-MoE-4x11b</h1>
    </div>
    <div class="info">
        <img src="https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/MHmVGOLGh4I5MfQ83iiXS.jpeg">
        <p><strong>Creator:</strong> <a href="https://huggingface.co/Steelskull" target="_blank">SteelSkull</a></p>
        <p><strong>About Umbra-v3-MoE-4x11b:</strong> A Mixture of Experts model designed for general assistance with a special knack for storytelling and RP/ERP</p>
        <p>Integrates models from notable sources for enhanced performance in diverse tasks.</p>
        <p><strong>Source Models:</strong></p>
        <ul>
            <li><a href="https://huggingface.co/Himitsui/Kaiju-11B">Himitsui/Kaiju-11B</a></li>
            <li><a href="https://huggingface.co/Sao10K/Fimbulvetr-11B-v2">Sao10K/Fimbulvetr-11B-v2</a></li>
            <li><a href="https://huggingface.co/decapoda-research/Antares-11b-v2">decapoda-research/Antares-11b-v2</a></li>
            <li><a href="https://huggingface.co/beberik/Nyxene-v3-11B">beberik/Nyxene-v3-11B</a></li>
        </ul>
    </div>
    <div class="update-section">
        <h2>Update-Log:</h2>
        <p>The [Umbra Series] keeps rolling out from the [Lumosia Series] garage, aiming to be your digital Alfred with a side of Shakespeare for those RP/ERP nights.</p>
        <p><strong>What's Fresh in v3?</strong></p>
        <p>Didn’t reinvent the wheel, just slapped on some fancier rims. Upgraded the models and tweaked the prompts a bit. Now, Umbra's not just a general use LLM; it's also focused on spinning stories and "Stories".</p>
        <p><strong>Negative Prompt Minimalism</strong></p>
        <p>Got the prompts to do a bit of a diet and gym routine—more beef on the positives, trimming down the negatives as usual with a dash of my midnight musings.</p>
        <p><strong>Still Guessing, Aren’t We?</strong></p>
        <p>Just so we're clear, "v3" is not the messiah of updates. It’s another experiment in the saga.</p>
        <p>Dive into Umbra v3 and toss your two cents my way. Your feedback is the caffeine in my code marathon.</p>
        <p><strong></strong>Exl2 available by:</strong></p>
        <p>EXL2-Rpcal = <a href="https://huggingface.co/AzureBlack/Umbra-v3-MoE-4x11b-exl2-8bpw-8h-rpcal" target="_blank">AzureBlack</p>
        <p></p>
        <p>GGUF = <a href="https://huggingface.co/mradermacher/Umbra-v3-MoE-4x11b-GGUF" target="_blank">mradermacher</p>
    </div>
</div>
</body>
</html>

# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Steelskull__Umbra-v3-MoE-4x11b)

|             Metric              |Value|
|---------------------------------|----:|
|Avg.                             |73.09|
|AI2 Reasoning Challenge (25-Shot)|68.43|
|HellaSwag (10-Shot)              |87.83|
|MMLU (5-Shot)                    |65.99|
|TruthfulQA (0-shot)              |69.30|
|Winogrande (5-shot)              |83.90|
|GSM8k (5-shot)                   |63.08|