Spaces:

hoduyquocbao
/

LLAMA3.2-GRop

Running on Zero

hoduyquocbao commited on about 13 hours ago

Commit

c3f15f3

•

1 Parent(s): 4bf04ee

fix device_map

Files changed (1) hide show

app.py CHANGED Viewed

@@ -19,7 +19,7 @@ tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
 model = AutoModelForCausalLM.from_pretrained(
     MODEL_ID,
     torch_dtype=torch.float16,
-    device_map="cuda",
     low_cpu_mem_usage=True
 ).eval()

 model = AutoModelForCausalLM.from_pretrained(
     MODEL_ID,
     torch_dtype=torch.float16,
+    device_map="auto",
     low_cpu_mem_usage=True
 ).eval()