The success of deep convolutional neural networks has raised the question of what representations are learned by neurons located in intermediate and deep layers. One approach towards understanding how CNNs work and learn is to visualize individual neurons' activations and receptive fields.
Zeiler and Fergus [1]} proposed a deconvolution network in order to visualize the original patterns that activate the corresponding activation maps. A deconvolution network consists of the three steps of unpooling, transposed convolution, and the ReLU operation.
Yosinski  [2]} developed two tools for understanding deep convolutional neural networks. The first of these tools is designed to visualize the activation maps at different layers for a given input image. The second tool aims to estimate the input pattern which a network is maximally attuned to for a given object class. In practice, the last layer of a deep neural network typically consists of one neuron per object class. Yosinski proposed to use gradient ascent (with regularization) to find the input image that maximizes the output of a specific neuron in regard to a specific object class. Hence, they derive the optimum input that appeals to the network for a specific class.
