size mismatch for model.layers when loading with from pretrained
Hello Community,
When I want to try out the model and landing it with from pretrained(), I face:
RuntimeError Traceback (most recent call last) Cell In[5], line 1----> 1 model = AutoModelForCausalLM.from_pretrained(model_id) 2 tokenizer = AutoTokenizer.from_pretrained(model_id) File ~/code/genai-ml/.venv/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:563, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs) 561 elif type(config) in cls._model_mapping.keys(): 562 model_class = _get_model_class(config, cls._model_mapping) --> 563 return model_class.from_pretrained( 564 pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs 565 ) 566 raise ValueError( 567 f"Unrecognized configuration class {config.class} for this kind of AutoModel: {cls.name}.\n" 568 f"Model type should be one of {', '.join(c.name for c in cls._model_mapping.keys())}." 569 ) File ~/code/genai-ml/.venv/lib/python3.10/site-packages/transformers/modeling_utils.py:3754, in PreTrainedModel.from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, *model_args, **kwargs) 3744 if dtype_orig is not None: 3745 torch.set_default_dtype(dtype_orig) 3747 ( 3748 model, 3749 missing_keys, 3750 unexpected_keys, 3751 mismatched_keys,
...
size mismatch for model.layers.39.self_attn.q_proj.weight: copying a param with shape torch.Size([4096, 5120]) from checkpoint, the shape in current model is torch.Size([5120, 5120]). size mismatch for model.layers.39.self_attn.k_proj.weight: copying a param with shape torch.Size([1024, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]). size mismatch for model.layers.39.self_attn.v_proj.weight: copying a param with shape torch.Size([1024, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]). size mismatch for model.layers.39.self_attn.o_proj.weight: copying a param with shape torch.Size([5120, 4096]) from checkpoint, the shape in current model is torch.Size([5120, 5120]). You may consider adding ignore_mismatched_sizes=True
in the model from_pretrained
method.
Do you have any idea what is the reason for that?
kindest regards
I've faced the same issue.
When a new version (> 4.24.4) has been released :) https://github.com/huggingface/transformers/releases
Hi
@Xenova
,
do you know when we can install simply over pip install transformers? https://pypi.org/project/transformers/#history seems that current version is 4.43 which is above your stated 4.24
regards :)
Sorry that’s a typo, I meant 4.42.4!
Hi thanks but still: this means that the current release 443 over pip should be sufficient ? :) regards and thank you very much for your fast response and support
It should work, yes!