nvidia

/

NV-Embed-v1

like 408

Follow

NVIDIA 3,334

nvm it's not working

Also getting this error.

Thank you for reporting this issue. Can you try to install the required packages as below?

pip uninstall -y transformer-engine
pip install torch==2.2.0
pip install transformers --upgrade
pip install flash-attn==2.2.0
pip install sentence-transformers==2.7.0

Try loading the model like this:

        text_config = {"_name_or_path": "mistralai/Mistral-7B-v0.1"}
        self.model = AutoModel.from_pretrained(
            "nvidia/NV-Embed-v1",
            trust_remote_code=True,
            text_config=text_config,
        )

I have the same issue.

        _hf_model = AutoModel.from_pretrained('nvidia/NV-Embed-v1',
                                              trust_remote_code=True,
                                              text_config=text_config,
                                              use_auth_token=_HF_TOKEN)

results in this error:

Traceback (most recent call last):
  File "/dev-volume/vitruvius-backend/shared/room_transformer/text_embeddings.py", line 69, in <module>
    main()
  File "/dev-volume/vitruvius-backend/shared/room_transformer/text_embeddings.py", line 60, in main
    embedding = text_embedding_hf(args.text, args.device)
  File "/dev-volume/vitruvius-backend/shared/room_transformer/text_embeddings.py", line 39, in text_embedding_hf
    model = _get_hf_model(device)
  File "/dev-volume/vitruvius-backend/shared/room_transformer/text_embeddings.py", line 29, in _get_hf_model
    _hf_model = AutoModel.from_pretrained('nvidia/NV-Embed-v1',
  File "/dev-volume/.cache/pypoetry/virtualenvs/vitruvius-backend-CzLWYIoL-py3.10/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 558, in from_pretrained
    return model_class.from_pretrained(
  File "/dev-volume/.cache/pypoetry/virtualenvs/vitruvius-backend-CzLWYIoL-py3.10/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3626, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
  File "/dev-volume/data/.huggingface/modules/transformers_modules/nvidia/NV-Embed-v1/497fec6bd98c747cc5dad1c5ff5ef3d94dc0de8c/modeling_nvembed.py", line 326, in __init__
    self.tokenizer = AutoTokenizer.from_pretrained(config.text_config._name_or_path) if config.text_config is not None else None
  File "/dev-volume/.cache/pypoetry/virtualenvs/vitruvius-backend-CzLWYIoL-py3.10/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 837, in from_pretrained
    config = AutoConfig.from_pretrained(
  File "/dev-volume/.cache/pypoetry/virtualenvs/vitruvius-backend-CzLWYIoL-py3.10/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 934, in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/dev-volume/.cache/pypoetry/virtualenvs/vitruvius-backend-CzLWYIoL-py3.10/lib/python3.10/site-packages/transformers/configuration_utils.py", line 632, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/dev-volume/.cache/pypoetry/virtualenvs/vitruvius-backend-CzLWYIoL-py3.10/lib/python3.10/site-packages/transformers/configuration_utils.py", line 689, in _get_config_dict
    resolved_config_file = cached_file(
  File "/dev-volume/.cache/pypoetry/virtualenvs/vitruvius-backend-CzLWYIoL-py3.10/lib/python3.10/site-packages/transformers/utils/hub.py", line 417, in cached_file
    raise EnvironmentError(
OSError: You are trying to access a gated repo.
Make sure to have access to it at https://huggingface.co/mistralai/Mistral-7B-v0.1.
401 Client Error. (Request ID: Root=1-66686085-49367655425d427772650e58;2819bcda-07ae-4c58-969a-e6124e1ae416)

Cannot access gated repo for url https://huggingface.co/mistralai/Mistral-7B-v0.1/resolve/main/config.json.
Access to model mistralai/Mistral-7B-v0.1 is restricted. You must be authenticated to access it.

You need to (1) to accept the conditions of mistralai/Mistral-7B-v0.1 and (2) set the HF_TOKEN environment variable.

I'm getting the same access error.
I have access to mistralai/Mistral-7B-v0.1 and can manually download the config.json file.
HF_TOKEN is set

I find the similar issue in mistralai: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1/discussions/2
Here are two possible solutions for this issue.

1. installing transformers from source as follows

pip install --upgrade git+https://github.com/huggingface/transformers

2. Try to updated Transformers and renamed "mistral" to "llama" in config.json file.

I am having the same error too, yet none of the solutions above work. I want to try renaming mistral to llama in the config.json file but I am unsure how to proceed. The only keys with mistral in them are "architectures": ["MistralModel"] and "model_type": "bidir_mistral". Do I replace the values of these keys? If so, what value do I replace them to?

Thanks!