Running sample code has a shape error:
#23
by
yingliuhf
- opened
I'm running the sample code
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("google/gemma-7b-it")
model = AutoModelForCausalLM.from_pretrained("google/gemma-7b-it", device_map="auto")
input_text = "Write me a poem about Machine Learning."
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))
But it's giving me a shape error:
RuntimeError: shape '[1, 9, 3072]' is invalid for input of size 36864
Yup, doesn't work with transformers.
This comment has been hidden
Try, it helped me, pip install "torch>=2.1.1" -U
Worked after updating the torch version.
Thanks! We'll fix it for lower torch versions. See https://huggingface.co/google/gemma-7b/discussions/17#65d64073550694a180cfd5df
osanseviero
changed discussion status to
closed
Also now pip install -U transformers if you cannot support latest versions of torch . We had a patch for this!