Ba et. al [1]} found that one way to reduce training time is to regularize the activity of neurons by computing the normalized mean and variance used to sum the inputs to neurons in a layer from all the summed inputs in a single training situation , also giving each neuron its own adaptive bias and gain, which are applied after normalization and before nonlinearity; layer normalization is very effective for stabilizing hidden state dynamics in recurrent networks.
