Update README.md
Browse files
README.md
CHANGED
@@ -99,15 +99,15 @@ model-index:
|
|
99 |
|
100 |
This is a new kind of model optimization. It is based on a new method for the analysis of the functional role of layers within the transformer stack, and on layer duplication (self-merging) to increase intelligence.
|
101 |
|
102 |
-
|
103 |
|
104 |
-
### Model improvement
|
105 |
| | Average | IFEval | BBH | MATH Lvl 5 | GPQA | MUSR | MMLU-PRO |
|
106 |
|-----------------|---------|--------|------|------------|------|-------|----------|
|
107 |
-
| RYS Improvement |
|
108 |
|
109 |
|
110 |
-
This model is based on MaziyarPanahi/calme-2.1-qwen2-72b, which was tuned from Qwen2-72B. As this method is orthogonal to fine-tuning, the further finetune from MaziyarPanahi now has the top position:
|
111 |
https://huggingface.co/MaziyarPanahi/calme-2.4-rys-78b
|
112 |
|
113 |
|
@@ -117,7 +117,7 @@ This research was supported with hardware from the [appliedAI Institute](https:/
|
|
117 |
|
118 |
## Quickstart
|
119 |
|
120 |
-
Here is a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate
|
121 |
|
122 |
```python
|
123 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
99 |
|
100 |
This is a new kind of model optimization. It is based on a new method for the analysis of the functional role of layers within the transformer stack, and on layer duplication (self-merging) to increase intelligence.
|
101 |
|
102 |
+
### No Weights were modified in this process!
|
103 |
|
104 |
+
### Model improvement with layer duplication:
|
105 |
| | Average | IFEval | BBH | MATH Lvl 5 | GPQA | MUSR | MMLU-PRO |
|
106 |
|-----------------|---------|--------|------|------------|------|-------|----------|
|
107 |
+
| RYS Improvement | 2.61% | -2.05% |2.51% | 8.16% |2.58% |17.72% | 0.31% |
|
108 |
|
109 |
|
110 |
+
This model is based on MaziyarPanahi/calme-2.1-qwen2-72b, which in turn was tuned from Qwen2-72B. As this method is orthogonal to fine-tuning, the further finetune from MaziyarPanahi now has the top position:
|
111 |
https://huggingface.co/MaziyarPanahi/calme-2.4-rys-78b
|
112 |
|
113 |
|
|
|
117 |
|
118 |
## Quickstart
|
119 |
|
120 |
+
Here is a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate content.
|
121 |
|
122 |
```python
|
123 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|