Quantization made by Richard Erkhov.
yi-bagel-2x34b - GGUF
- Model creator: https://huggingface.co/NLPinas/
- Original model: https://huggingface.co/NLPinas/yi-bagel-2x34b/
| Name | Quant method | Size |
|---|---|---|
| yi-bagel-2x34b.Q2_K.gguf | Q2_K | 11.94GB |
| yi-bagel-2x34b.IQ3_XS.gguf | IQ3_XS | 13.26GB |
| yi-bagel-2x34b.IQ3_S.gguf | IQ3_S | 13.99GB |
| yi-bagel-2x34b.Q3_K_S.gguf | Q3_K_S | 13.93GB |
| yi-bagel-2x34b.IQ3_M.gguf | IQ3_M | 14.5GB |
| yi-bagel-2x34b.Q3_K.gguf | Q3_K | 15.51GB |
| yi-bagel-2x34b.Q3_K_M.gguf | Q3_K_M | 15.51GB |
| yi-bagel-2x34b.Q3_K_L.gguf | Q3_K_L | 16.89GB |
| yi-bagel-2x34b.IQ4_XS.gguf | IQ4_XS | 17.36GB |
| yi-bagel-2x34b.Q4_0.gguf | Q4_0 | 18.13GB |
| yi-bagel-2x34b.IQ4_NL.gguf | IQ4_NL | 18.3GB |
| yi-bagel-2x34b.Q4_K_S.gguf | Q4_K_S | 18.25GB |
| yi-bagel-2x34b.Q4_K.gguf | Q4_K | 19.24GB |
| yi-bagel-2x34b.Q4_K_M.gguf | Q4_K_M | 12.05GB |
| yi-bagel-2x34b.Q4_1.gguf | Q4_1 | 17.51GB |
| yi-bagel-2x34b.Q5_0.gguf | Q5_0 | 17.49GB |
| yi-bagel-2x34b.Q5_K_S.gguf | Q5_K_S | 21.55GB |
| yi-bagel-2x34b.Q5_K.gguf | Q5_K | 22.65GB |
| yi-bagel-2x34b.Q5_K_M.gguf | Q5_K_M | 22.65GB |
| yi-bagel-2x34b.Q5_1.gguf | Q5_1 | 24.05GB |
| yi-bagel-2x34b.Q6_K.gguf | Q6_K | 26.28GB |
| yi-bagel-2x34b.Q8_0.gguf | Q8_0 | 34.03GB |
Original model description:
base_model: - jondurbin/bagel-dpo-34b-v0.2 - jondurbin/nontoxic-bagel-34b-v0.2 tags: - mergekit - merge license: other license_name: yi-license license_link: https://huggingface.co/01-ai/Yi-34B-200K/blob/main/LICENSE
yi-bagel-2x34b
Released January 11, 2024
This is a merge of pre-trained language models created using mergekit. For more information, kindly refer to the model cards from jondurbin linked in the section below. This model debuted in the leaderboard at rank #4 (January 11, 2024).
Merge Details
Merge Method
This model is an expertimental merge using the linear merge method. This is to assess the degree of which the DPO has an effect, in terms of censoring, as used in jondurbin/bagel-dpo-34b-v0.2.
Models Merged
The following models were included in the merge:
Open LLM Leaderboard Metrics (as of January 11, 2024)
| Metric | Value |
|---|---|
| MMLU (5-shot) | 76.60 |
| ARC (25-shot) | 72.70 |
| HellaSwag (10-shot) | 85.44 |
| TruthfulQA (0-shot) | 71.42 |
| Winogrande (5-shot) | 82.72 |
| GSM8K (5-shot) | 60.73 |
| Average | 74.93 |
According to the leaderboard description, here are the benchmarks used for the evaluation:
- MMLU (5-shot) - a test to measure a text model’s multitask accuracy. The test covers 57 tasks including elementary mathematics, US history, computer science, law, and more.
- AI2 Reasoning Challenge -ARC- (25-shot) - a set of grade-school science questions.
- HellaSwag (10-shot) - a test of commonsense inference, which is easy for humans (~95%) but challenging for SOTA models.
- TruthfulQA (0-shot) - a test to measure a model’s propensity to reproduce falsehoods commonly found online.
- Winogrande (5-shot) - an adversarial and difficult Winograd benchmark at scale, for commonsense reasoning.
- GSM8k (5-shot) - diverse grade school math word problems to measure a model's ability to solve multi-step mathematical reasoning problems.
Configuration
The following YAML configuration was used to produce this model:
models:
- model: jondurbin/nontoxic-bagel-34b-v0.2
parameters:
weight: 0.5
- model: jondurbin/bagel-dpo-34b-v0.2
parameters:
weight: 0.5
merge_method: linear
dtype: float16
Further Information
For additional information or inquiries about yi-bagel-2x34b, please contact the developer through email: [email protected].
- Downloads last month
- 389