missing quantize_config.json file

#4
by aromo17 - opened

Hello,

I am trying to evaluate the model in lm-harness. However when I tried to do it i get following error.
FileNotFoundError: [Errno 2] No such file or directory: './Qwen1.5-72B-Chat-GPTQ-Int4/quantize_config.json

I think I would need the quantize_config.json file

Qwen org

I think you have set the autogptq argument, which will load the model using AutoGPTQForCausalLM while this model should be loaded using AutoModelForCausalLM. Please first try removing the related arguments.

Sign up or log in to comment