---
tags:
- meta-llama/Meta-Llama-3-8B
---

# Maverick Llama 3 8b 🏴‍☠️




model is using ChatML prompt template format.

example:

```
<|im_start|>system
You are Maverick, a helpful AI assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

```

🏆 Evaluation

## eq-bench 

| Tasks  |Version|Filter|n-shot|     Metric      | Value |   |Stderr|
|--------|------:|------|-----:|-----------------|------:|---|-----:|
|eq_bench|    2.1|none  |     0|eqbench          |56.8710|±  |2.8946|
|        |       |none  |     0|percent_parseable|99.4152|±  |0.5848|

## winogrande 

|  Tasks   |Version|Filter|n-shot|Metric|Value |   |Stderr|
|----------|------:|------|-----:|------|-----:|---|-----:|
|winogrande|      1|none  |     0|acc   |0.7372|±  |0.0124|

## gsm8k

|Tasks|Version|     Filter     |n-shot|  Metric   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|-----:|---|-----:|
|gsm8k|      3|strict-match    |     5|exact_match|0.5694|±  |0.0136|
|     |       |flexible-extract|     5|exact_match|0.5701|±  |0.0136|

## mathqa 

|Tasks |Version|Filter|n-shot| Metric |Value |   |Stderr|
|------|------:|------|-----:|--------|-----:|---|-----:|
|mathqa|      1|none  |     0|acc     |0.3930|±  |0.0089|
|      |       |none  |     0|acc_norm|0.3869|±  |0.0089|

This model is based on Llama-3-8b, and is governed by [META LLAMA 3 COMMUNITY LICENSE AGREEMENT](LICENSE)