Weight size VS VRAM requirements
#8
by
mindkrypted
- opened
Hello, I'd be interested to hear about what makes the model's inference require the large amount of VRAM ( 4 x 80GB )?
Weight is about 40GB and logically fits into 48GB of VRAM with enough leftover for a short* context.
Very promising model, great job from the team.
Thanks!