The simplest example is to have faster/slower learning rates in the upper/lower layers of a network. I found this post on tensorflow. https://stackoverflow.com/questions/34945554/how-to-set-layer-wise-learning-rate-in-tensorflowFirst question is whether similar trick possible in keras. My second question is whether one could go one step further, set different learning rates for a specific range/set of neurons/weights in a particular layer.
possible to use different learning rate for different neuron in keras/tensorflow?
deep learning;tensorflow;keras;training
You can follow this thread here: https://github.com/fchollet/keras/pull/3004 Seems at least that the layer-wise tuning should come.