gurgutan commited on
Commit
2f0aa8b
1 Parent(s): b28d871

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -8
README.md CHANGED
@@ -6,17 +6,14 @@ language:
6
  ---
7
 
8
  # Saiga2-13B-4bit
9
- This files are GPTQ model files for [saiga2-13B-lora](https://huggingface.co/IlyaGusev/saiga2_13b_lora) model.
10
-
11
  ## Technical details
12
- Model was quantized to 4-bit with [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ) library
13
-
14
  ## Examples of usage
15
- First make sure you have [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ) installed:
16
-
17
  GITHUB_ACTIONS=true pip install auto-gptq
18
 
19
- Then try the following example code:
20
 
21
  ```python
22
  from transformers import AutoTokenizer, TextGenerationPipeline
@@ -82,7 +79,6 @@ MODEL_NAME = "gurgutan/saiga2-13b-4bit"
82
  DEFAULT_MESSAGE_TEMPLATE = "<s>{role}\n{content}</s>\n"
83
  DEFAULT_SYSTEM_PROMPT = "Ты — Сайга, русскоязычный автоматический ассистент. Ты разговариваешь с людьми и помогаешь им."
84
 
85
-
86
  tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME, use_fast=True)
87
  model = AutoGPTQForCausalLM.from_quantized(MODEL_NAME, device="cuda:0", use_safetensors=True, use_triton=False)
88
  generation_config = GenerationConfig.from_pretrained(MODEL_NAME)
 
6
  ---
7
 
8
  # Saiga2-13B-4bit
9
+ Это GPTQ модель для [saiga2-13B-lora](https://huggingface.co/IlyaGusev/saiga2_13b_lora) model.
 
10
  ## Technical details
11
+ Модель квантизована в 4-битную с помощью [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ) library
 
12
  ## Examples of usage
13
+ Удостоверьтесь, что [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ) установлена:
 
14
  GITHUB_ACTIONS=true pip install auto-gptq
15
 
16
+ Пример кода для использования модели в генерации ответа:
17
 
18
  ```python
19
  from transformers import AutoTokenizer, TextGenerationPipeline
 
79
  DEFAULT_MESSAGE_TEMPLATE = "<s>{role}\n{content}</s>\n"
80
  DEFAULT_SYSTEM_PROMPT = "Ты — Сайга, русскоязычный автоматический ассистент. Ты разговариваешь с людьми и помогаешь им."
81
 
 
82
  tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME, use_fast=True)
83
  model = AutoGPTQForCausalLM.from_quantized(MODEL_NAME, device="cuda:0", use_safetensors=True, use_triton=False)
84
  generation_config = GenerationConfig.from_pretrained(MODEL_NAME)