In a neural network, each neuron will have it's activation.But what the activation mean?Does it just mean nothing but a temporal value to produce the final result or it's has something to with our understanding to the problem?For example, considering a neural network to recognize a handwritten character from a picture, I wonder if it's possible that a neuron's activation represents the how the pixel matches the specific value in a small part of the picture?
What does the activation of a neuron mean?
neural network
The activation of a neuron is mathematically nothing but a function of its input. Consider a neural network with one hidden layer and one input vector $\mathbf{x}$. The input $a_j$ of neuron $j$ can be written as:  $a_j = w_j^T \mathbf{x} + b_j$. The activation of neuron $j$ is then a transformation $g: \mathbf{R} \rightarrow \mathbf{R}$ of the input. For exmaple, one can use the sigmoid activiation function$sig(a_j) = \dfrac{1}{1 + \exp(-a_j)} $The weights $w$ are chosen to minimize a loss function and not for the sake of their interpretation. Nevertheless, one can try to interpret the activation of a neuron as internal representation of the input. One of the nicest examples I've seen comes from the Rummelhart et al (1986), Figure 4. In that paper, two family trees were fed into a neural network. The family trees represented an Italian and an English speaking family comprising three generations. Among other things, when activating one name of the tree, the neuronal activities represented whether the person was from the English or Italian tree and which generation the person embodied.   