Can the yi-34b.Q3_K_S.gguf model be set up on Mac M1 with 16GB of memory?

#7
by liweixin - opened

Can the yi-34b.Q3_K_S.gguf model be set up on Mac M1 with 16GB of memory?

M1 with 16GB memory can use up to ±67% of it as VRAM. The largest model available is up to 20B Q3_K_S with 1500 context.

Sign up or log in to comment