FileNotFoundError: Could not find model in TheBloke/Nous-Hermes-13B-GPTQ
#18
by
synapse
- opened
I am getting the error:FileNotFoundError: Could not find model in TheBloke/Nous-Hermes-13B-GPTQ
Am I missing something in my code below (running on collab)?:
model_name_or_path = "TheBloke/Nous-Hermes-13B-GPTQ"
model_basename = "nous-hermes-13b-GPTQ-4bit-128g.no-act.order"
use_triton = False
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=True)
model = AutoGPTQForCausalLM.from_quantized(model_name_or_path,
model_basename=model_basename,
use_safetensors=True,
trust_remote_code=True,
device="cuda:0",
use_triton=use_triton,
quantize_config=None)
+1
seems to be gone?