f16 support
#6
by
jrsimuix
- opened
When trying Phi 3.5 I'm running into ''f16' type used without 'f16' extension enabled'
Checking https://webgpufundamentals.org/webgpu/lessons/webgpu-limits-and-features.html
I can see unfortunately running chrome Canary 131.0.6750.0 that shader-f16 is not available on by default adapter, I tried enabling the chrome flag for experimental without success.
Would you be able to add some checks with transformers to notify users that their system is not compatible before loading the model?
Also I noticed you had a dtype: "q4", without f16 commented out in your demo worker but looking in files I didn't see this listed to try to fallback - is it possible to use PHI without f16 or the performance would be too bad?
Many thanks!! :)