Commit
•
71566a7
1
Parent(s):
e015e6d
Update README.md
Browse files
README.md
CHANGED
|
@@ -12,12 +12,18 @@ tags:
|
|
| 12 |
## Model Architecture
|
| 13 |
The Mistral-7B-v0.1 model is a transformer-based model that can handle a variety of tasks including but not limited to translation, summarization, and text completion. It's particularly designed for the Italian language and can be fine-tuned for specific tasks.
|
| 14 |
|
|
|
|
| 15 |
## Evaluation
|
| 16 |
-
[Leaderboard Ita LLM](https://huggingface.co/spaces/FinancialSupport/open_ita_llm_leaderboard)
|
| 17 |
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
|
| 23 |
**Quantized 4-Bit Version Available**
|
|
|
|
| 12 |
## Model Architecture
|
| 13 |
The Mistral-7B-v0.1 model is a transformer-based model that can handle a variety of tasks including but not limited to translation, summarization, and text completion. It's particularly designed for the Italian language and can be fine-tuned for specific tasks.
|
| 14 |
|
| 15 |
+
|
| 16 |
## Evaluation
|
|
|
|
| 17 |
|
| 18 |
+
For a detailed comparison of model performance, check out the [Leaderboard for Italian Language Models](https://huggingface.co/spaces/FinancialSupport/open_ita_llm_leaderboard).
|
| 19 |
+
|
| 20 |
+
Here's a breakdown of the performance metrics:
|
| 21 |
+
|
| 22 |
+
| Metric | hellaswag_it acc_norm | arc_it acc_norm | m_mmlu_it 5-shot acc | Average |
|
| 23 |
+
|:----------------------------|:----------------------|:----------------|:---------------------|:--------|
|
| 24 |
+
| **Accuracy Normalized** | 0.6734 | 0.5466 | 0.5334 | 0.5844 |
|
| 25 |
+
|
| 26 |
+
---
|
| 27 |
|
| 28 |
|
| 29 |
**Quantized 4-Bit Version Available**
|