Hugging Face's logo

mpasila
/
Finnish-Alpaca-Tiny-V2-7B

Text Generation
Transformers
Safetensors
Finnish
llama
text-generation-inference
unsloth
trl
sft
Inference Endpoints
Model card Files Community
Finnish-Alpaca-Tiny-V2-7B
File size: 6,843 Bytes 
067f16e
 
545c052
067f16e
 
 
 
 
 
 
fcceba6
067f16e
545c052
 
067f16e
545c052
 
 
 
 
 
7c0708b
534f669
545c052
 
 
 
 
 
 
 
 
067f16e
7c0708b
 
 
 
467b282
 
7c0708b
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
067f16e
 
 
 
 
 
 
 
545c052
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
 
---
language:
- fi
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
- sft
base_model: LumiOpen/Viking-7B
datasets:
- mpasila/Finnish-Alpaca-Tiny
---
This is a merge of [mpasila/Finnish-Alpaca-Tiny-V2-LoRA-7B](https://huggingface.co/mpasila/Finnish-Alpaca-Tiny-V2-LoRA-7B).

LoRA trained in 4-bit with 2k context using [LumiOpen/Viking-7B](https://huggingface.co/LumiOpen/Viking-7B/) as the base model for 1 epoch.

Dataset used is [mpasila/Finnish-Alpaca-Tiny](https://huggingface.co/datasets/mpasila/Finnish-Alpaca-Tiny).

It works relatively well for question and answering. I will make a bigger dataset for the next fine-tune.

### Prompt format: Alpaca
It uses Alpaca format but with a translated instruction at the start:
```
{
    "instruction,output": "Alla on ohje, jossa kuvataan tehtävä. Kirjoita vastaus, joka täyttää pyynnön asianmukaisesti.\n\n### Instruction:\n%instruction%\n\n### Response:\n%output%",
    "instruction,input,output": "Alla on ohje, jossa kuvataan tehtävä ja joka on yhdistetty kontekstia lisäävään syötteeseen. Kirjoita vastaus, joka täyttää pyynnön asianmukaisesti.\n\n### Instruction:\n%instruction%\n\n### Input:\n%input%\n\n### Response:\n%output%"
}

```

## Evaluation

| Model | Size | Type | FIN-bench (score) |
|-------|------|------|-------|
| **mpasila/Finnish-Alpaca-Tiny-V2-7B** | 7B | Instruct | **0.4654** |
| [mpasila/Finnish-Alpaca-Small-7B](https://huggingface.co/mpasila/Finnish-Alpaca-Small-7B) | 7B | Instruct | 0.3586 |
| [mpasila/Alpacazord-Viking-7B](https://huggingface.co/mpasila/Alpacazord-Viking-7B) | 7B | Instruct | 0.4123 |
| [mpasila/NordicAlpaca-Finnish-V1-7B](https://huggingface.co/mpasila/NordicAlpaca-Finnish-V1-7B) | 7B | Instruct | 0.3891 |
| [mpasila/Finnish-Viking-Alpaca-V1-7B](https://huggingface.co/mpasila/Finnish-Viking-Alpaca-V1-7B) | 7B | Instruct | 0.3943 |
| [Finnish-NLP/llama-7b-finnish-instruct-v0.1](https://huggingface.co/Finnish-NLP/llama-7b-finnish-instruct-v0.1) | 7B | Instruct | 0.4365 |
| [Finnish-NLP/llama-7b-finnish-instruct-v0.2](https://huggingface.co/Finnish-NLP/llama-7b-finnish-instruct-v0.2) | 7B | Instruct | 0.3993 |
| [Finnish-NLP/llama-7b-finnish](https://huggingface.co/Finnish-NLP/llama-7b-finnish) | 7B | Base | 0.2350 |
| [LumiOpen/Viking-7B (1000B)](https://huggingface.co/LumiOpen/Viking-7B) | 7B | Base | 0.3721 |
| [HPLT/gpt-7b-nordic-prerelease](https://huggingface.co/HPLT/gpt-7b-nordic-prerelease) | 7B | Base | 0.3169 |

[Source](https://docs.google.com/spreadsheets/d/1rqJb9dQVihg-Z1_Ras1L_-wuzPg9xNzpdmM2x5HueeY/edit?usp=sharing)

#### FIN-bench scores:

|                      Task                      |Version|       Metric        |Value |   |Stderr|
|------------------------------------------------|------:|---------------------|-----:|---|-----:|
|bigbench_analogies                              |      0|multiple_choice_grade|0.6385|±  |0.0423|
|bigbench_arithmetic_1_digit_addition            |      0|multiple_choice_grade|0.7200|±  |0.0451|
|bigbench_arithmetic_1_digit_division            |      0|multiple_choice_grade|0.7391|±  |0.0936|
|bigbench_arithmetic_1_digit_multiplication      |      0|multiple_choice_grade|0.4800|±  |0.0502|
|bigbench_arithmetic_1_digit_subtraction         |      0|multiple_choice_grade|0.6300|±  |0.0485|
|bigbench_arithmetic_2_digit_addition            |      0|multiple_choice_grade|0.4000|±  |0.0492|
|bigbench_arithmetic_2_digit_division            |      0|multiple_choice_grade|0.5000|±  |0.0503|
|bigbench_arithmetic_2_digit_multiplication      |      0|multiple_choice_grade|0.2800|±  |0.0451|
|bigbench_arithmetic_2_digit_subtraction         |      0|multiple_choice_grade|0.4300|±  |0.0498|
|bigbench_arithmetic_3_digit_addition            |      0|multiple_choice_grade|0.5800|±  |0.0496|
|bigbench_arithmetic_3_digit_division            |      0|multiple_choice_grade|0.3100|±  |0.0465|
|bigbench_arithmetic_3_digit_multiplication      |      0|multiple_choice_grade|0.2900|±  |0.0456|
|bigbench_arithmetic_3_digit_subtraction         |      0|multiple_choice_grade|0.5100|±  |0.0502|
|bigbench_arithmetic_4_digit_addition            |      0|multiple_choice_grade|0.5300|±  |0.0502|
|bigbench_arithmetic_4_digit_division            |      0|multiple_choice_grade|0.3900|±  |0.0490|
|bigbench_arithmetic_4_digit_multiplication      |      0|multiple_choice_grade|0.3100|±  |0.0465|
|bigbench_arithmetic_4_digit_subtraction         |      0|multiple_choice_grade|0.6200|±  |0.0488|
|bigbench_arithmetic_5_digit_addition            |      0|multiple_choice_grade|0.6500|±  |0.0479|
|bigbench_arithmetic_5_digit_division            |      0|multiple_choice_grade|0.3200|±  |0.0469|
|bigbench_arithmetic_5_digit_multiplication      |      0|multiple_choice_grade|0.3000|±  |0.0461|
|bigbench_arithmetic_5_digit_subtraction         |      0|multiple_choice_grade|0.6400|±  |0.0482|
|bigbench_cause_and_effect_one_sentence          |      0|multiple_choice_grade|0.5686|±  |0.0700|
|bigbench_cause_and_effect_one_sentence_no_prompt|      0|multiple_choice_grade|0.6471|±  |0.0676|
|bigbench_cause_and_effect_two_sentences         |      0|multiple_choice_grade|0.4314|±  |0.0700|
|bigbench_emotions                               |      0|multiple_choice_grade|0.2250|±  |0.0331|
|bigbench_empirical_judgments                    |      0|multiple_choice_grade|0.2525|±  |0.0439|
|bigbench_general_knowledge                      |      0|multiple_choice_grade|0.3429|±  |0.0571|
|bigbench_hhh_alignment_harmless                 |      0|multiple_choice_grade|0.3793|±  |0.0643|
|bigbench_hhh_alignment_helpful                  |      0|multiple_choice_grade|0.3390|±  |0.0622|
|bigbench_hhh_alignment_honest                   |      0|multiple_choice_grade|0.3729|±  |0.0635|
|bigbench_hhh_alignment_other                    |      0|multiple_choice_grade|0.5349|±  |0.0770|
|bigbench_intent_recognition                     |      0|multiple_choice_grade|0.2153|±  |0.0156|
|bigbench_misconceptions                         |      0|multiple_choice_grade|0.5224|±  |0.0433|
|bigbench_paraphrase                             |      0|multiple_choice_grade|0.4750|±  |0.0354|
|bigbench_sentence_ambiguity                     |      0|multiple_choice_grade|0.4833|±  |0.0651|
|bigbench_similarities_abstraction               |      0|multiple_choice_grade|0.6974|±  |0.0530|

# Uploaded  model

- **Developed by:** mpasila
- **License:** apache-2.0
- **Finetuned from model :** LumiOpen/Viking-7B

This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)