Unsupervised pre-training was discovered to provide a superior alternative to fully stochastic parameter initialization of deep neural networks [1]}, [2]}, [3]}.
The use of end-to-end pre-training on image data has been widely studied due to its potential benefits [4]}, [5]}, typically using the pixel-wise L2/MSE reconstruction loss for optimization.
In this paper, we explore PSE as a location-sensitive image reconstruction loss for unsupervised pre-training with images, and examine if this can lead to better feature learning.
