AIDC-AI
/

Ovis1.6-Llama3.2-3B-GPTQ-Int4

Image-Text-to-Text

text-generation

4-bit precision

Model card Files Files and versions Community

TryingHard commited on 6 days ago

Commit

d29d9f7

•

1 Parent(s): 45291da

Update README.md

Files changed (1) hide show

README.md +1 -0

README.md CHANGED Viewed

@@ -199,6 +199,7 @@ model = OvisLlamaGPTQForCausalLM.from_pretrained(
     llm_attn_implementation='eager',
     trust_remote_code=True
 ).cuda()
 print(f"Model Loaded!")

     llm_attn_implementation='eager',
     trust_remote_code=True
 ).cuda()
+model.model.llm.model.config.use_cache = False
 print(f"Model Loaded!")