Llama-3.1-8B-Instruct

Runtime error

vilarin commited on Jul 23

Commit

6f2eb6f

•

1 Parent(s): 677d853

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -40,6 +40,7 @@ model = AutoModelForCausalLM.from_pretrained(
     MODEL,
     torch_dtype=torch.bfloat16,
     device_map="auto",
     quantization_config=quantization_config)
 @spaces.GPU()

     MODEL,
     torch_dtype=torch.bfloat16,
     device_map="auto",
+    rope_scaling={"type": "llama3", "factor": 8.0},
     quantization_config=quantization_config)
 @spaces.GPU()