saurabhhssaurabh
commited on
Commit
•
5318b4c
1
Parent(s):
4fc53f3
Update chat template for RakutenAI-7B-Chat model
Browse filesAdded custom chat template for RakutenAI-7B-Chat. It allows user to use "tokenizer.apply_chat_template()" function to convert input chat data into required format. Please go through the example provided in ReadME file for more detail.
README.md
CHANGED
@@ -16,6 +16,37 @@ An independent evaluation by Kamata et.al. for [Nejumi LLMリーダーボード
|
|
16 |
## Usage
|
17 |
|
18 |
```python
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
19 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
20 |
|
21 |
model_path = "Rakuten/RakutenAI-7B-chat"
|
|
|
16 |
## Usage
|
17 |
|
18 |
```python
|
19 |
+
|
20 |
+
# With RakutenAI-7B-Chat's custom chat template.
|
21 |
+
|
22 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
23 |
+
|
24 |
+
model_path = "Rakuten/RakutenAI-7B-chat"
|
25 |
+
tokenizer = AutoTokenizer.from_pretrained(model_path)
|
26 |
+
model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype="auto", device_map="auto")
|
27 |
+
model.eval()
|
28 |
+
|
29 |
+
chat = [
|
30 |
+
|
31 |
+
{"role": "system", "content": "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions."},
|
32 |
+
{"role": "user", "content": "How to make an authentic Spanish Omelette?"},
|
33 |
+
]
|
34 |
+
|
35 |
+
input_ids = tokenizer.apply_chat_template(chat, tokenize=True, add_generation_prompt=True, return_tensors="pt").to(device=model.device)
|
36 |
+
tokens = model.generate(
|
37 |
+
input_ids,
|
38 |
+
max_length=4096,
|
39 |
+
do_sample=False,
|
40 |
+
num_beams=1,
|
41 |
+
pad_token_id=tokenizer.eos_token_id,
|
42 |
+
)
|
43 |
+
out = tokenizer.decode(tokens[0][len(input_ids[0]):], skip_special_tokens=True)
|
44 |
+
print("ASSISTANT:\n" + out)
|
45 |
+
print()
|
46 |
+
|
47 |
+
|
48 |
+
# Without using custom chat template.
|
49 |
+
|
50 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
51 |
|
52 |
model_path = "Rakuten/RakutenAI-7B-chat"
|