One of the challenges of deep learning is that the gradients with respect to the weights in a layer are highly dependent on the outputs of neurons in the previous layer, especially if these outputs vary in a highly correlated way. The Batch normalization [1]} method normalizes the summed input of each hidden unit on the training instance. Specifically, for the input in the ith layer, the batch normalization method resizes the input according to the variance under the data distribution, according to the sum of the data.
