chinoll commited on
Commit
925c5fb
1 Parent(s): 8867475

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -1
README.md CHANGED
@@ -9,4 +9,30 @@ datasets:
9
  library_name: transformers
10
  ---
11
 
12
- base Yi-6B-200k
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
  library_name: transformers
10
  ---
11
 
12
+ ## Examples
13
+ ```python
14
+ from transformers import AutoModelForCausalLM, AutoTokenizer
15
+
16
+ model_path = 'chinoll/Yi-6b-200k-dpo'
17
+
18
+ tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=False)
19
+
20
+ # Since transformers 4.35.0, the GPT-Q/AWQ model can be loaded using AutoModelForCausalLM.
21
+ model = AutoModelForCausalLM.from_pretrained(
22
+ model_path,
23
+ device_map="auto",
24
+ torch_dtype='auto'
25
+ ).eval()
26
+
27
+ # Prompt content: "hi"
28
+ messages = [
29
+ {"role": "user", "content": "hi"}
30
+ ]
31
+
32
+ input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, add_generation_prompt=True, return_tensors='pt')
33
+ output_ids = model.generate(input_ids.to('cuda'))
34
+ response = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=True)
35
+
36
+ # Model response: "Hello! How can I assist you today?"
37
+ print(response)
38
+ ```