llm-jp
/

llm-jp-13b-dpo-lora-hh_rlhf_ja-v1.1

Text Generation

Model card Files Files and versions Community

Taka008 commited on Mar 12

Commit

da802b0

•

1 Parent(s): 524b999

Update README.md

Files changed (1) hide show

README.md +4 -2

README.md CHANGED Viewed

@@ -63,9 +63,11 @@ Checkpoints format: Hugging Face Transformers (Megatron-DeepSpeed format models
 ```python
 import torch
-from transformers import AutoTokenizer, AutoModelForCausalLM
 tokenizer = AutoTokenizer.from_pretrained("llm-jp/llm-jp-13b-dpo-lora-hh_rlhf_ja-v1.1")
-model = AutoModelForCausalLM.from_pretrained("llm-jp/llm-jp-13b-dpo-lora-hh_rlhf_ja-v1.1", device_map="auto", torch_dtype=torch.float16)
 text = "以下は、タスクを説明する指示です。要求を適切に満たす応答を書きなさい。\n\n### 指示:\n{instruction}\n\n### 応答:\n".format(instruction="自然言語処理とは何か")
 tokenized_input = tokenizer.encode(text, add_special_tokens=False, return_tensors="pt").to(model.device)
 with torch.no_grad():

 ```python
 import torch
+from transformers import AutoTokenizer
+from peft import AutoPeftModelForCausalLM
 tokenizer = AutoTokenizer.from_pretrained("llm-jp/llm-jp-13b-dpo-lora-hh_rlhf_ja-v1.1")
+model = AutoPeftModelForCausalLM.from_pretrained("llm-jp/llm-jp-13b-dpo-lora-hh_rlhf_ja-v1.1", device_map="auto", torch_dtype=torch.float16)
 text = "以下は、タスクを説明する指示です。要求を適切に満たす応答を書きなさい。\n\n### 指示:\n{instruction}\n\n### 応答:\n".format(instruction="自然言語処理とは何か")
 tokenized_input = tokenizer.encode(text, add_special_tokens=False, return_tensors="pt").to(model.device)
 with torch.no_grad():