How to use llama 3sft model, pipeline or tokenizer.apply_chat_template. Can you provide a simple example? Thank you very much for your contribution
How to use llama 3sft model, pipeline or tokenizer.apply_chat_template. Can you provide a simple example? Thank you very much for your contribution
pipeline:pipeline = transformers.pipeline(
"text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto"
tokenizer:message = [
{"role": "user",
"content": t_prompt},
]
could you open an issue at the RLHFlow GitHub project?
could you open an issue at the RLHFlow GitHub project?
I have found the issue and fixed it. I discovered that the eos_topken_id in the tokenizer and the eos_token_id in the generation were inconsistent in RLHFlow/LLaMA3-SFT, with values of 128009 and 128001, respectively. After unifying the eos_token_id, I was able to get a proper response using the template from Llama 3 DPO.