SakuraLLM

/

Sakura-13B-LNovel-v0_8-4bit

like 2

Follow

SakuraLLM 89

Traceback (most recent call last):
File "E:\FFOutput\App\Sakura-13B-Galgame\translate_novel.py", line 248, in
main()
File "E:\FFOutput\App\Sakura-13B-Galgame\translate_novel.py", line 195, in main
sakura_model = M.SakuraModel(cfg=cfg)
File "E:\FFOutput\App\Sakura-13B-Galgame\utils\model.py", line 135, in init
(tokenizer, model) = load_model(cfg)
File "E:\FFOutput\App\Sakura-13B-Galgame\utils\model.py", line 84, in load_model
model = AutoGPTQForCausalLM.from_quantized(
File "E:\FFOutput\App\Sakura-13B-Galgame\sakura\lib\site-packages\auto_gptq\modeling\auto.py", line 108, in from_quantized
return quant_func(
File "E:\FFOutput\App\Sakura-13B-Galgame\sakura\lib\site-packages\auto_gptq\modeling_base.py", line 923, in from_quantized
model = autogptq_post_init(model, use_act_order=quantize_config.desc_act)
File "E:\FFOutput\App\Sakura-13B-Galgame\sakura\lib\site-packages\auto_gptq\modeling_utils.py", line 258, in autogptq_post_init
prepare_buffers(device, buffers["temp_state"], buffers["temp_dq"])
RuntimeError: no device index

E:\FFOutput\App\Sakura-13B-Galgame\sakura\python.exe translate_novel.py --model_name_or_path E:\FFOutput\App\Sakura-13B-Galgame\models\SakuraLLM\Sakura-13B-LNovel-v0_8-4bit --trust_remote_code --model_version 0.8 --use_gptq_model --text_length 512 --data_path novel.txt --output_path novel_translated.txt