INT4 model shows bad perf than FP32 on Intel CPU,why?

#13

by Sakura10151 - opened Jul 19, 2023

Jul 19, 2023

int4 model log shows it will consume hundreds of times more time than the fp32 model on SelfAttention

Dec 8, 2023

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment