where \(x\)  is the input of size \((b,k,l_j)\) . The mean and standard-deviation of \(x\)  are vectors of size \(l_j\)  calculated over the mini-batches and all the \(k\)  colors (in order to keep the invariance property). \(\gamma \)  and \(\mu \)  are learnable parameter in \(\mathbb {R}\) . \(\epsilon =10^{-5}\)  is a value added to the denominator for numerical stability. As in the original work of [1]}, during training the batch normalization layer keeps running estimates of its computed mean and variance, which are then used for normalization during evaluation. The running estimates are kept with a default momentum of 0.1.
