The fact that the ABA and ABB structures containing both
seen and unseen words are handled effectively implies that
the knowledge embodied in the connection weights has been,
to some extent, abstracted away from the particular
examples seen during training.
[1]} interpret the behaviour of the infants
in their experiments
as similarly implicating some form of abstract knowledge.
In particular, they suggest that the infants in their
experiment had learned algebra-like rules
that represent relationships between placeholders (variables),
such as `the first item X is the same as the third item Y'.
While this mechanism for representing the structure of the
sequences, in terms of using variables, is clearly distinct
from the convolutional approach employed here,
there are, nonetheless, underlying similarities.
When, for instance, the network processes the sequence
wo fe wo, the wo channels contain the values
101, representing the fact that this particular word
occurs at the beginning and end of the sequence.
Moreover, from the point of view of filters
that do not discriminate between words,
these values simply signal that the same item
occurs in both first and third position,
which is what [1]} suggest the infants
are sensitive to.
