An Elman recurrent neural network is a Simple Recurrent Network (SRN) where the hidden layer activation values are copied (during each time step) into the context layer. In the following time step, this context layer is used as an input to the hidden layer. In this way, even if all the other inputs stay the same, the network can produce a different output (based on its previous activation status) and is able to learn sequential tasks [1]}.
There are more complex and powerful recurrent neural network currently used, such as Long Short-Term Memory (LSTM) networks. These networks are designed to learn long-term dependencies in the input sequence [2]}. Learning these long-term dependencies is difficult to obtain with classical SRNs. However, in the case of our problem, we used only up to 15 times steps sequences, and the following step in the sequence can be determined (in the case of counting words) directly from the previous one. In this scenario, the use of a more complex structure is unnecessary to obtain the counting and pointing behaviour. Elman-type networks were successfully used for counting purpose before by [3]} to count letters, by [4]}, [5]} to count elements in the simplified visual input and by [6]} to count number words.
