Layer-wise Model Freezing –
Layer freezing (dropping) is the representative layer-wise technique for neural network training ([1]}, [2]}, [3]}, [4]}).
All these methods commonly stop updating the parameters of the layers in a bottom-up direction.
These empirical techniques are supported by the analysis presented in ([5]}).
Since the layers converge from the input-side sequentially, the layer-wise freezing can reduce the training time without strongly affecting the accuracy.
These previous works clearly demonstrate the advantages of processing individual layers separately.
