saurabhhssaurabh commited on
Commit
5318b4c
1 Parent(s): 4fc53f3

Update chat template for RakutenAI-7B-Chat model

Browse files

Added custom chat template for RakutenAI-7B-Chat. It allows user to use "tokenizer.apply_chat_template()" function to convert input chat data into required format. Please go through the example provided in ReadME file for more detail.

Files changed (1) hide show
  1. README.md +31 -0
README.md CHANGED
@@ -16,6 +16,37 @@ An independent evaluation by Kamata et.al. for [Nejumi LLMリーダーボード
16
  ## Usage
17
 
18
  ```python
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
  from transformers import AutoModelForCausalLM, AutoTokenizer
20
 
21
  model_path = "Rakuten/RakutenAI-7B-chat"
 
16
  ## Usage
17
 
18
  ```python
19
+
20
+ # With RakutenAI-7B-Chat's custom chat template.
21
+
22
+ from transformers import AutoModelForCausalLM, AutoTokenizer
23
+
24
+ model_path = "Rakuten/RakutenAI-7B-chat"
25
+ tokenizer = AutoTokenizer.from_pretrained(model_path)
26
+ model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype="auto", device_map="auto")
27
+ model.eval()
28
+
29
+ chat = [
30
+
31
+ {"role": "system", "content": "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions."},
32
+ {"role": "user", "content": "How to make an authentic Spanish Omelette?"},
33
+ ]
34
+
35
+ input_ids = tokenizer.apply_chat_template(chat, tokenize=True, add_generation_prompt=True, return_tensors="pt").to(device=model.device)
36
+ tokens = model.generate(
37
+ input_ids,
38
+ max_length=4096,
39
+ do_sample=False,
40
+ num_beams=1,
41
+ pad_token_id=tokenizer.eos_token_id,
42
+ )
43
+ out = tokenizer.decode(tokens[0][len(input_ids[0]):], skip_special_tokens=True)
44
+ print("ASSISTANT:\n" + out)
45
+ print()
46
+
47
+
48
+ # Without using custom chat template.
49
+
50
  from transformers import AutoModelForCausalLM, AutoTokenizer
51
 
52
  model_path = "Rakuten/RakutenAI-7B-chat"