I had a simple neural network that was outputting the same value regardless of the input. During training, it was behaving normally, with training and validation loss diminishing to a floor value.The data range was in [-1000, +1000].The model is a 1D convolutional network. If it is useful :     train_loss = lcategorical_crossentropy(...).mean()    net = InputLayer((None, net_z, net_x), input_var=input_var)    net = ConvLayer(net,  16, 9, pad='same' )#, flip_filters=False)    net = ConvLayer(net,  16, 9, pad='same')#, flip_filters=False)    net = DenseLayer(net,   num_units=32)    net = DropoutLayer(net, p=0.5)    net = DenseLayer(net, num_units=2, nonlinearity=None)    net = NonlinearityLayer(net, softmax)This was corrected by normalizing the data in [-1, 1]What is the reason behind that ? Backpropagation being stuck I guess, but where can I learn about that in depth ?
Neural Network outputing the same value / normalization
machine learning;neural network;backpropagation;normalization