* For \(N=10K\), we use \(wd=0.02\), \(lr=0.001/0.002\), and batch size 24 with fp16;
* For \(N=20K\), we use \(wd=0.02\), \(lr=0.001/0.002\), and batch size 48 with fp16;
* For \(N=50K\), we use \(wd=0.02\), \(lr=0.001/0.002\), and batch size 96 with fp16;
* For \(N=100K,200K\), we use \(wd=0.02\), \(lr=0.001/0.002\), batch size 192 with fp16;
* For \(N=500K,1M\), we use \(wd=0.01\), \(lr=0.0005/0.001\), batch size 192 with fp16;
* For \(N=2M\), we use \(wd=0.005\), \(lr=0.002\), and batch size 1536 with fp16;
* For \(N=5M\), we use \(wd=0.002\), \(lr=0.0005\), and batch size 1536 with fp16;
* For \(N=10M\), we use \(wd=0.001\), \(lr=0.0005\), and batch size 1536 with fp16.