A comparative study of back propagation and its alternatives on multilayer perceptrons
The de facto algorithm for training the back pass of a feedforward neural
network is backpropagation (BP). The use of almost-everywhere differentiable
activation functions made it efficient and effective to propagate the gradient
backwards through layers of deep neural networks. However, in recent years,
there has been much research in alternatives to backpropagation. This analysis
has largely focused on reaching state-of-the-art accuracy in multilayer
perceptrons (MLPs) and convolutional neural networks (CNNs). In this paper, we
analyze the stability and similarity of predictions and neurons in MLPs and
propose a new variation of one of the algorithms.