In SRCNN, researchers find that better reconstruction performance can be obtained by adding more convolutional layers to increase the receptive field. However, directly stacking the layers will cause vanishing/exploding gradients and degradation problem [1]}. Meanwhile, adding more layers will lead to a higher training error and more expensive computational cost.
