Marvin Minsky Seymour Papert multilayer perceptrons deep learning Alexey Grigorevich Ivakhnenko deep learning stochastic gradient descent Shun'ichi Amari internal representations perceptron exclusive-or
Some say that neural network research stagnated after the publication of machine learning research by Marvin Minsky and Seymour Papert (1969). They discovered two key issues with the computational machines that processed neural networks. The first issue was that single-layer neural networks were incapable of processing the exclusive-or circuit. The second significant issue was that computers were not sophisticated enough to effectively handle the long run time required by large neural networks. However, by the time this book came out, methods for training multilayer perceptrons (MLPs) were already known. The first deep learning MLP was published by Alexey Grigorevich Ivakhnenko and Valentin Lapa in 1965. The first deep learning MLP trained by stochastic gradient descent was published in 1967 by Shun'ichi Amari.
In computer experiments conducted by Amari's student Saito, a five layer MLP with two modifiable layers learned useful internal representations to classify non-linearily separable pattern classes.
