Text Generation
Transformers
PyTorch
RefinedWeb
sft
custom_code
text-generation-inference

Which Azure VM can run the model?

#6
by AbRds - opened

Hi everyone,

We are thinking on running this model in an Azure VM, but we are not sure which one to choose. Somebody has tested the model on Azure?
We have three possible VM, any recommendation?

NC24ads_A100_v4 24vCPU, 220GB RAM, 80GB GPU Memory
NC64as_T4_v3 64vCPU, 440GB RAM, 64GB GPU Memory
ND24s 24vCPU, 440GB RAM, 96GB GPU Memory

Thanks in advance

Sign up or log in to comment