FP32 or FP16
#20
by
timbmg
- opened
Hi,
I am wondering whether to use fp32 or fp16. In the config.json, fp32 is set.
However, in the paper (page 5), it says:
After training, we directly take the last checkpoint for evaluation. We run model training on up to 8 NVIDIA A100 GPUs with 80GB memory and model evaluation on up to 8 NVIDIA Tesla V100 GPUs with 32GB memory. Models are trained with mixed precision using fp16 and evaluated with half precision fp16 as well.
This is also the configuration for MTEB in the Readme.
We recommend using fp16 for model inference.