backpropagation Leibniz chain rule automatic differentiation reverse accumulation Seppo Linnainmaa Frank Rosenblatt Henry J. Kelley control theory Paul Werbos nodes
Neural network research was boosted when computers achieved greater processing power. Also key in later advances was the backpropagation algorithm. It is an efficient application of the Leibniz chain rule (1673) to networks of differentiable nodes. It is also known as 
the reverse mode of automatic differentiation or reverse accumulation, due to Seppo Linnainmaa (1970). The term "back-propagating errors" was introduced in 1962 by Frank Rosenblatt, but he did not have an implementation of this procedure, although Henry J. Kelley had a continuous precursor of backpropagation already in 1960 in the context of control theory. In 1982, Paul Werbos applied backpropagation to MLPs in the way that has become standard.
