| Quantization made by Richard Erkhov. | |
| [Github](https://github.com/RichardErkhov) | |
| [Discord](https://discord.gg/pvy7H8DZMG) | |
| [Request more models](https://github.com/RichardErkhov/quant_request) | |
| gemma-2b-orpo - bnb 8bits | |
| - Model creator: https://huggingface.co/anakin87/ | |
| - Original model: https://huggingface.co/anakin87/gemma-2b-orpo/ | |
| Original model description: | |
| --- | |
| license: other | |
| license_name: gemma-terms-of-use | |
| license_link: https://ai.google.dev/gemma/terms | |
| library_name: transformers | |
| base_model: google/gemma-2b | |
| tags: | |
| - trl | |
| - orpo | |
| - generated_from_trainer | |
| model-index: | |
| - name: gemma-2b-orpo | |
| results: | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: AI2 Reasoning Challenge (25-Shot) | |
| type: ai2_arc | |
| config: ARC-Challenge | |
| split: test | |
| args: | |
| num_few_shot: 25 | |
| metrics: | |
| - type: acc_norm | |
| value: 49.15 | |
| name: normalized accuracy | |
| source: | |
| url: >- | |
| https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=anakin87%2Fgemma-2b-orpo | |
| name: Open LLM Leaderboard | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: HellaSwag (10-Shot) | |
| type: hellaswag | |
| split: validation | |
| args: | |
| num_few_shot: 10 | |
| metrics: | |
| - type: acc_norm | |
| value: 73.72 | |
| name: normalized accuracy | |
| source: | |
| url: >- | |
| https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=anakin87%2Fgemma-2b-orpo | |
| name: Open LLM Leaderboard | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: MMLU (5-Shot) | |
| type: cais/mmlu | |
| config: all | |
| split: test | |
| args: | |
| num_few_shot: 5 | |
| metrics: | |
| - type: acc | |
| value: 38.52 | |
| name: accuracy | |
| source: | |
| url: >- | |
| https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=anakin87%2Fgemma-2b-orpo | |
| name: Open LLM Leaderboard | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: TruthfulQA (0-shot) | |
| type: truthful_qa | |
| config: multiple_choice | |
| split: validation | |
| args: | |
| num_few_shot: 0 | |
| metrics: | |
| - type: mc2 | |
| value: 44.53 | |
| source: | |
| url: >- | |
| https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=anakin87%2Fgemma-2b-orpo | |
| name: Open LLM Leaderboard | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: Winogrande (5-shot) | |
| type: winogrande | |
| config: winogrande_xl | |
| split: validation | |
| args: | |
| num_few_shot: 5 | |
| metrics: | |
| - type: acc | |
| value: 64.33 | |
| name: accuracy | |
| source: | |
| url: >- | |
| https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=anakin87%2Fgemma-2b-orpo | |
| name: Open LLM Leaderboard | |
| - task: | |
| type: text-generation | |
| name: Text Generation | |
| dataset: | |
| name: GSM8k (5-shot) | |
| type: gsm8k | |
| config: main | |
| split: test | |
| args: | |
| num_few_shot: 5 | |
| metrics: | |
| - type: acc | |
| value: 13.87 | |
| name: accuracy | |
| source: | |
| url: >- | |
| https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=anakin87%2Fgemma-2b-orpo | |
| name: Open LLM Leaderboard | |
| datasets: | |
| - alvarobartt/dpo-mix-7k-simplified | |
| language: | |
| - en | |
| --- | |
| <img src="./assets/gemma-2b-orpo.png" width="450"></img> | |
| # gemma-2b-orpo | |
| This is an ORPO fine-tune of [google/gemma-2b](https://huggingface.co/google/gemma-2b) with | |
| [`alvarobartt/dpo-mix-7k-simplified`](https://huggingface.co/datasets/alvarobartt/dpo-mix-7k-simplified). | |
| **⚡ Quantized version (GGUF)**: https://huggingface.co/anakin87/gemma-2b-orpo-GGUF | |
| ## ORPO | |
| [ORPO (Odds Ratio Preference Optimization)](https://arxiv.org/abs/2403.07691) is a new training paradigm that combines the usually separated phases | |
| of SFT (Supervised Fine-Tuning) and Preference Alignment (usually performed with RLHF or simpler methods like DPO). | |
| - Faster training | |
| - Less memory usage (no reference model needed) | |
| - Good results! | |
| ## 🏆 Evaluation | |
| ### Nous | |
| gemma-2b-orpo performs well for its size on Nous' benchmark suite. | |
| (evaluation conducted using [LLM AutoEval](https://github.com/mlabonne/llm-autoeval)). | |
| | Model | Average | AGIEval | GPT4All | TruthfulQA | Bigbench | | |
| |---|---:|---:|---:|---:|---:| | |
| | [**anakin87/gemma-2b-orpo**](https://huggingface.co/anakin87/gemma-2b-orpo) [📄](./assets/gemma-2b-orpo-Nous.md) | **39.45** | 23.76 | 58.25 | 44.47 | 31.32 | | |
| | [mlabonne/Gemmalpaca-2B](https://huggingface.co/mlabonne/Gemmalpaca-2B) [📄](https://gist.github.com/mlabonne/4b638752fc3227df566f9562064cb864) | 38.39 | 24.48 | 51.22 | 47.02 | 30.85 | | |
| | [google/gemma-2b-it](https://huggingface.co/google/gemma-2b-it) [📄](https://gist.github.com/mlabonne/db0761e74175573292acf497da9e5d95) | 36.1 | 23.76 | 43.6 | 47.64 | 29.41 | | |
| | [google/gemma-2b](https://huggingface.co/google/gemma-2b) [📄](https://gist.github.com/mlabonne/7df1f238c515a5f63a750c8792cef59e) | 34.26 | 22.7 | 43.35 | 39.96 | 31.03 | | |
| ### [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) | |
| Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_anakin87__gemma-2b-orpo). | |
| By comparison, on the Open LLM Leaderboard, google/gemma-2b-it has an average of 42.75. | |
| | Metric |Value| | |
| |---------------------------------|----:| | |
| |Avg. |47.35| | |
| |AI2 Reasoning Challenge (25-Shot)|49.15| | |
| |HellaSwag (10-Shot) |73.72| | |
| |MMLU (5-Shot) |38.52| | |
| |TruthfulQA (0-shot) |44.53| | |
| |Winogrande (5-shot) |64.33| | |
| |GSM8k (5-shot) |13.87| | |
| ## 🙏 Dataset | |
| [`alvarobartt/dpo-mix-7k-simplified`](https://huggingface.co/datasets/alvarobartt/dpo-mix-7k-simplified) | |
| is a simplified version of [`argilla/dpo-mix-7k`](https://huggingface.co/datasets/argilla/dpo-mix-7k). | |
| You can find more information [in the dataset card](https://huggingface.co/datasets/alvarobartt/dpo-mix-7k-simplified). | |
| ## 🎮 Model in action | |
| ### Usage notebook | |
| [📓 Chat and RAG using Haystack](./notebooks/usage.ipynb) | |
| ### Simple text generation with Transformers | |
| The model is small, so it runs smoothly on Colab. *It is also fine to load the model using quantization*. | |
| ```python | |
| # pip install transformers accelerate | |
| import torch | |
| from transformers import pipeline | |
| pipe = pipeline("text-generation", model="anakin87/gemma-2b-orpo", torch_dtype=torch.bfloat16, device_map="auto") | |
| messages = [{"role": "user", "content": "Write a rap song on Vim vs VSCode."}] | |
| prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False) | |
| outputs = pipe(prompt, max_new_tokens=500, do_sample=True, temperature=0.7, top_k=50, top_p=0.95) | |
| print(outputs[0]["generated_text"]) | |
| ``` | |
| ## Training | |
| The model was trained using HF TRL. | |
| [📓 Training notebook](./notebooks/training.ipynb) | |
| ### Framework versions | |
| - Transformers 4.39.1 | |
| - Pytorch 2.2.0+cu121 | |
| - Datasets 2.18.0 | |
| - Tokenizers 0.15.2 | |