Post
538
π€ I did some test of AI performance of
@Intel
Ultra 9 185h GPU and NPU!
(My RAM: LPDDR5x 7467 16GB * 2)
πΊ GPU:
π» @OpenVINO :
rajatkrishna/Meta-Llama-3-8B-OpenVINO-INT4 : 12 tok/s
π»IPEX-LLM (Llama.cpp backend):
Qwen/Qwen2-7B-Instruct-GGUF Q4_K_M: 15.5 tok/s
Qwen/Qwen2-0.5B-Instruct-GGUF Q8_0: 65 tok/s
π§ π€ NPU:
microsoft/Phi-3-mini-128k-instruct int4: 7 tok/s
π€It looks like it have really reached a usable level!
(My RAM: LPDDR5x 7467 16GB * 2)
πΊ GPU:
π» @OpenVINO :
rajatkrishna/Meta-Llama-3-8B-OpenVINO-INT4 : 12 tok/s
π»IPEX-LLM (Llama.cpp backend):
Qwen/Qwen2-7B-Instruct-GGUF Q4_K_M: 15.5 tok/s
Qwen/Qwen2-0.5B-Instruct-GGUF Q8_0: 65 tok/s
π§ π€ NPU:
microsoft/Phi-3-mini-128k-instruct int4: 7 tok/s
π€It looks like it have really reached a usable level!
