size mismatch for model.layers when loading with from pretrained

#35
by C-Felix - opened

Hello Community,
When I want to try out the model and landing it with from pretrained(), I face:


RuntimeError Traceback (most recent call last) Cell In[5], line 1----> 1 model = AutoModelForCausalLM.from_pretrained(model_id) 2 tokenizer = AutoTokenizer.from_pretrained(model_id) File ~/code/genai-ml/.venv/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:563, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs) 561 elif type(config) in cls._model_mapping.keys(): 562 model_class = _get_model_class(config, cls._model_mapping) --> 563 return model_class.from_pretrained( 564 pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs 565 ) 566 raise ValueError( 567 f"Unrecognized configuration class {config.class} for this kind of AutoModel: {cls.name}.\n" 568 f"Model type should be one of {', '.join(c.name for c in cls._model_mapping.keys())}." 569 ) File ~/code/genai-ml/.venv/lib/python3.10/site-packages/transformers/modeling_utils.py:3754, in PreTrainedModel.from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, *model_args, **kwargs) 3744 if dtype_orig is not None: 3745 torch.set_default_dtype(dtype_orig) 3747 ( 3748 model, 3749 missing_keys, 3750 unexpected_keys, 3751 mismatched_keys,
...
size mismatch for model.layers.39.self_attn.q_proj.weight: copying a param with shape torch.Size([4096, 5120]) from checkpoint, the shape in current model is torch.Size([5120, 5120]). size mismatch for model.layers.39.self_attn.k_proj.weight: copying a param with shape torch.Size([1024, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]). size mismatch for model.layers.39.self_attn.v_proj.weight: copying a param with shape torch.Size([1024, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]). size mismatch for model.layers.39.self_attn.o_proj.weight: copying a param with shape torch.Size([5120, 4096]) from checkpoint, the shape in current model is torch.Size([5120, 5120]). You may consider adding ignore_mismatched_sizes=True in the model from_pretrained method.


Do you have any idea what is the reason for that?
kindest regards

I've faced the same issue.

Hi there! As stated in the README, you need to install Transformers from source using:

pip install git+https://github.com/huggingface/transformers.git

Hi @Xenova , Thank you for this hint. Do you have any idea when simple pip install transformers is sufficient in future? regards :)

When a new version (> 4.24.4) has been released :) https://github.com/huggingface/transformers/releases

Hi @Xenova ,
do you know when we can install simply over pip install transformers? https://pypi.org/project/transformers/#history seems that current version is 4.43 which is above your stated 4.24
regards :)

Sorry that’s a typo, I meant 4.42.4!

Hi thanks but still: this means that the current release 443 over pip should be sufficient ? :) regards and thank you very much for your fast response and support

It should work, yes!

Sign up or log in to comment