eachadea

/

legacy-vicuna-13b

like 94

I've updated all of the files to match lmsys/vicuna-13b-delta-v0

Not sure what could be causing this. Are you getting the same error on the delta files?

@xerxes01 Which transformers version are you using?? I get the same error when using transformers version 4.27.4, in the FastChat repo its mentioned that we use transformers with a particular commit id. (Installed using this command "pip3 install git+https://github.com/huggingface/transformers.git@c612628045822f909020f7eb6784c79700813eda")
But i am unable to run inference on the model, @eachadea could you please link a colab notebook running the model?

if i use multiple gpus i keep getting TypeError: dispatch_model() got an unexpected keyword argument 'offload_index'

Had the same issue as xerxes01

same here followed the instruction and tried several things floating around even did a second clean setup. still an issue.

The token error should be related to this issue: https://github.com/huggingface/transformers/issues/22222

The token error should be related to this issue: https://github.com/huggingface/transformers/issues/22222

I'm not sure this is related, as that error was regarding LLaMATokenizer not existing (and suggesting to use LlamaTokenizer instead), whereas it looks like OP's error is complaining that LlamaTokenizer does not exist.

I also have this error with another form of Vicuna and updating transformers (or pinning it to previous commits) doesn't seem to have an effect. Stack trace for capitalization example:

(base) $ sudo docker run -p 80:80 -e MODEL=eachadea/vicuna-13b hyperonym/basaran:0.14.1
Downloading (…)okenizer_config.json: 100%|██████████| 727/727 [00:00<00:00, 107kB/s]
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/app/basaran/__main__.py", line 38, in <module>
    stream_model = load_model(
  File "/app/basaran/model.py", line 318, in load_model
    tokenizer = AutoTokenizer.from_pretrained(name_or_path, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/transformers/models/auto/tokenization_auto.py", line 676, in from_pretrained
    raise ValueError(
ValueError: Tokenizer class LlamaTokenizer does not exist or is not currently imported.

Did anyone find solution ?

facing the same issue, when loading FreeWilly model...
https://huggingface.co/stabilityai/FreeWilly2