Error while trying to loading the model: OpenFlamingo-9B-vitl-mpt7b model
###while I am trying to load the model using below code
from open_flamingo import create_model_and_transforms
model, image_processor, tokenizer = create_model_and_transforms(
clip_vision_encoder_path="ViT-L-14",
clip_vision_encoder_pretrained="openai",
lang_encoder_path="anas-awadalla/mpt-7b",
tokenizer_path="anas-awadalla/mpt-7b",
cross_attn_every_n_layers=4
)
ImportError: cannot import name '_expand_mask' from 'transformers.models.bloom.modeling_bloom' (/home/tele/.local/lib/python3.10/site-packages/transformers/models/bloom/modeling_bloom.py)
I am facing the same issue and it seems to be recent(loading 9b used to work before). Did you happen to resolve it?
cc: @anas-awadalla
The issue is caused by using recent transformers version. Downgrading transformers to transformers=4.28.1(the version used in open-flamingo) or other previous version solves the problem.
Hi @Nyandwi @saikiran7 . Yes I just tested on my end and looks like this issue is with the latest version of transformers. Thanks for pointing this out! I will open an issue on their repo.
Hi @anas-awadalla . Have you gotten a chance to raise the issue about the transformer version(in 9b model) or do you suggest a workaround in the meantime? Thanks :-)
