I got error when load this model with SentenceTransformer
I got error like:
File "/app/src/transformer_finetune/finetune.py", line 175, in __init__
self.model = SentenceTransformer(model_base, trust_remote_code=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 299, in __init__
modules = self._load_auto_model(
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 1324, in _load_auto_model
transformer_model = Transformer(
^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 54, in __init__
self._load_model(model_name_or_path, config, cache_dir, **model_args)
File "/usr/local/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 85, in _load_model
self.auto_model = AutoModel.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 567, in from_pretrained
raise ValueError(
ValueError: Unrecognized configuration class <class 'transformers_modules.vectara.hallucination_evaluation_model.2c44150de684dbcbc6eb2abd9e92b428c5fb3361.configuration_hhem_v2.HHEMv2Config'> for this kind of AutoModel: AutoModel.
Hi
@trinhhung
.
Thanks for your interest in HHEM. We apologize about the breaking change in HHEM-2.1 that it will no longer work SentenceTransformer.
Can you follow the new usage? In a few lines of code, you can use HHEM-21. with either AutoModel or pipeline, both of which are standard in the transformers lib.
@trinhhung
Despite the breaking change from HHEM-1.0-open to HHEM-2.1-Open, if you want to keep using HHEM-1.0 with sentence_transformers, just set revision to hhem-1.0-open when you load the model:
In [1]: from sentence_transformers import CrossEncoder
...: model = CrossEncoder('vectara/hallucination_evaluation_model', revision="hhem-1.0-open")
In [2]: scores = model.predict([
...: ["A man walks into a bar and buys a drink", "A bloke swigs alcohol at a pub"],
...: ["A person on a horse jumps over a broken down airplane.", "A person is at a diner, ordering an omelette."],
...: ["A person on a horse jumps over a broken down airplane.", "A person is outdoors, on a horse."],
...: ["A boy is jumping on skateboard in the middle of a red bridge.", "The boy skates down the sidewalk on a blue
...: bridge"],
...: ["A man with blond-hair, and a brown shirt drinking out of a public water fountain.", "A blond drinking water
...: in public."],
...: ["A man with blond-hair, and a brown shirt drinking out of a public water fountain.", "A blond man wearing a
...: brown shirt is reading a book."],
...: ["Mark Wahlberg was a fan of Manny.", "Manny was a fan of Mark Wahlberg."],
...: ])
...:
In [3]: scores
Out[3]:
array([6.1051613e-01, 4.7493548e-04, 9.9639291e-01, 2.1221593e-04,
9.9599433e-01, 1.4126928e-03, 2.8263088e-03], dtype=float32)
You might need a newer version of sentence_transformers to get the support for the revision argument as this feature was just added in Jan 2024. Simply run the command below to upgrade sentence_transformers on your computer:
pip3 install -U sentence_transformers