Since feedforward neural networks came into play in the mid 1980s, there have been many studies analysing and visualising such networks. Typically this has involved visualisation of decision boundaries, generalising from the linear boundaries of simple linear classifiers or perceptrons. Analysis of the benefits of multiple layers to the flexibility of decision boundaries played a part in a number of studies (e.g. [1]}), and was the focus of a number of general approximation proofs [2]}, [3]}, [4]}. These ideas were explored in key neural network texts [5]}. However the early adaptation of the sigmoid activation function hindered the visualisation of neural networks somewhat: the complex smooth non-linear surfaces given by sigmoidal neural networks even with few layers were very hard to decompose. Even where decision boundaries were illustrated, tracking these across the learning process was not done: there was more focus on the representation power of the neural network than a desire to follow the learning process. Nowadays, as people consider the unreasonable effectiveness of stochastic gradient methods (e.g. [6]}, [7]}), visualising learning processes can be insightful.
