Facing Problem with inference
There's always a mismatch between Expected size for first two dimensions of batch2 tensor
like execution of the following code gives a runtime error
inputs = tokenizer("<|prompter|>Hi how are you?<|endoftext|><|assistant|>", return_tensors="pt")
generation_output = model.generate(**inputs, return_dict_in_generate=True, output_scores=True)
RuntimeError: Expected size for first two dimensions of batch2 tensor to be: [40, 9] but got: [40, 1].
@Ron993 What is the rest of your code? I just tried the snippet you sent and it seemed to work just fine. Can you send a minimal representation that demonstrates your issue?
It may be possibly be versioning issues as well. Make sure everything is up to date
@ThatOneShortGuy Hi, Thanks for your feedback. It seems to work for me after updating the transformer version