|
--- |
|
language: |
|
- ru |
|
datasets: |
|
- IlyaGusev/saiga_scored |
|
- IlyaGusev/saiga_preferences |
|
license: apache-2.0 |
|
--- |
|
|
|
|
|
# Saiga/MistralNemo 12B, Russian fine-tune of Mistral Nemo |
|
|
|
Based on [an abliterated version](https://huggingface.co/natong19/Mistral-Nemo-Instruct-2407-abliterated) of [Mistral Nemo](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407). |
|
|
|
Llama.cpp version: TBD |
|
|
|
Colab: [link](https://colab.research.google.com/drive/1vNzMyPqx2GB7zk3ANDtZEfvhzgYOWu0B) |
|
|
|
## Prompt format |
|
|
|
Original Misral Nemo prompt format, but the system prompt is in the beginning: |
|
``` |
|
<s>Ты — Сайга, русскоязычный автоматический ассистент. Ты разговариваешь с людьми и помогаешь им. |
|
[INST]Как дела?[/INST] |
|
Отлично, а у тебя?</s> |
|
[INST]Шикарно. Как пройти в библиотеку?[/INST] |
|
``` |
|
|
|
## Code example |
|
```python |
|
# Исключительно ознакомительный пример. |
|
# НЕ НАДО ТАК ИНФЕРИТЬ МОДЕЛЬ В ПРОДЕ. |
|
# См. https://github.com/vllm-project/vllm или https://github.com/huggingface/text-generation-inference |
|
|
|
import torch |
|
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig |
|
|
|
MODEL_NAME = "IlyaGusev/saiga_nemo_12b" |
|
|
|
model = AutoModelForCausalLM.from_pretrained( |
|
MODEL_NAME, |
|
load_in_8bit=True, |
|
torch_dtype=torch.bfloat16, |
|
device_map="auto" |
|
) |
|
model.eval() |
|
|
|
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME) |
|
generation_config = GenerationConfig.from_pretrained(MODEL_NAME) |
|
print(generation_config) |
|
|
|
inputs = ["Почему трава зеленая?", "Сочини длинный рассказ, обязательно упоминая следующие объекты. Дано: Таня, мяч"] |
|
for query in inputs: |
|
prompt = tokenizer.apply_chat_template([{ |
|
"role": "user", |
|
"content": query |
|
}], tokenize=False, add_generation_prompt=True) |
|
data = tokenizer(prompt, return_tensors="pt", add_special_tokens=False) |
|
data = {k: v.to(model.device) for k, v in data.items()} |
|
data.pop("token_type_ids", None) |
|
output_ids = model.generate(**data, generation_config=generation_config)[0] |
|
output_ids = output_ids[len(data["input_ids"][0]):] |
|
output = tokenizer.decode(output_ids, skip_special_tokens=True).strip() |
|
print(query) |
|
print(output) |
|
print() |
|
print("==============================") |
|
print() |
|
``` |
|
|
|
## Output examples |
|
``` |
|
User: Почему трава зеленая? |
|
Saiga: TBD |
|
``` |
|
|
|
``` |
|
User: Сочини длинный рассказ, обязательно упоминая следующие объекты. Дано: Таня, мяч |
|
Saiga: TBD |
|
``` |
|
|
|
|
|
## Versions |
|
v1: |
|
- [87a83ce252ff0142cd4cc918fb3e6a9875ca4638](https://huggingface.co/IlyaGusev/saiga_nemo_12b/commit/87a83ce252ff0142cd4cc918fb3e6a9875ca4638) |
|
- Other name: saiga_nemo_12b_sft_m9_d14_simpo_m19_d31 |
|
- SFT dataset config: [sft_d14.json](https://github.com/IlyaGusev/saiga/blob/main/configs/datasets/sft_d14.json) |
|
- SFT model config: [saiga_nemo_12b_sft_m9.json](https://github.com/IlyaGusev/saiga/blob/main/configs/models/saiga_nemo_12b_sft_m9.json) |
|
- SimPO dataset config: [pref_d31.json](https://github.com/IlyaGusev/saiga/blob/main/configs/datasets/pref_d31.json) |
|
- SimPO model config: [saiga_nemo_12b_simpo_m19.json](https://github.com/IlyaGusev/saiga/blob/main/configs/models/saiga_nemo_12b_simpo_m19.json) |
|
- SFT wandb: [link](https://wandb.ai/ilyagusev/rulm_self_instruct/runs/e74ozfzh) |
|
- SimPO wandb: [link](https://wandb.ai/ilyagusev/rulm_self_instruct/runs/b094iiej) |
|
|
|
|
|
## Evaluation |
|
|
|
RuArenaHard: |
|
|
|
 |
|
|
|
|
|
PingPing: |
|
|
|
 |
|
|
|
|
