mosaicml

/

mpt-7b

like 1.16k

Follow

Mosaic ML, Inc. 352

Since the attention mask refactor in transformers main some private methods has been removed such as _expand_mask

https://github.com/huggingface/transformers/pull/27086

Loading the model fails with this error trace:

    from transformers.models.bloom.modeling_bloom import _expand_mask as _expand_mask_bloom
ImportError: cannot import name '_expand_mask' from 'transformers.models.bloom.modeling_bloom' 

Simple script to repro:

from accelerate import init_empty_weights
from transformers import AutoModelForCausalLM, AutoConfig

model_id = "mosaicml/mpt-7b"
config = AutoConfig.from_pretrained(
    model_id, trust_remote_code=True, revision="72e5f594ce36f9cabfa2a9fd8f58b491eb467ee7"
)
with init_empty_weights():
    model = AutoModelForCausalLM.from_config(
        config, trust_remote_code=True, code_revision="72e5f594ce36f9cabfa2a9fd8f58b491eb467ee7"
    )

Meaning a patch will be needed here for future transformers versions.