What is the problem with training Neural Networks with back propagation with activation functions that only output positive values?
Why are non zero-centered activation functions a problem in backpropagation?