normalization
With which std mean and var is the model normalized? I just want to know because in the internet there are several versions of it.
Mentioned for this model:
"mean": [ 0.4914, 0.4822, 0.4465 ],
"std": [ 0.2023, 0.1994, 0.201 ]
I calculated for myself:
dataset = datasets.CIFAR10(root='./data/cifar10', download=True, train=True, transform=transforms.ToTensor())
mean = dataset.data.astype(float).mean(axis=(0,1,2)) / 255
std = dataset.data.astype(float).std(axis=(0,1,2)) / 255
print("cifar10", np.round(mean, 4), np.round(std, 4))
cifar10 [0.4914 0.4822 0.4465] [0.247 0.2435 0.2616]
I suggest you consult the URL https://github.com/edadaltocg/detectors/blob/master/src/detectors/data/constants.py in which I list all the normalizing constants used.
Yes, your "std" is wrong.
See this discussion: https://github.com/kuangliu/pytorch-cifar/issues/16
The most important aspect of the normalizing constants is matching the ones used to train the model at inference time.
They may differ from the true data statistics without hurting performance. The idea of using these transformations is to force the weights and biases of the network to have values more or less in the range of a standard Gaussian distribution.
Said that, it is not uncommon to have models trained on ImageNet, for instance, that use average and std = [0.5, 0.5, 0.5].
So, it is ok if they differ.