Our experiments showed that a convolutional network
was able to
generalise successfully to unseen digits
without featural overlap between training and test instances.
Thus, one potential solution to the identity learning challenge
[1]} turns out to be a standard architectural
design that predates it by almost two decades
[2]}.
In concrete terms, weight sharing across
digit positions avoids the problem of training
independence identified by [1]},
allowing what is learned at seen positions
to be transferred to unseen positions.
More abstractly, symmetry provides a notion of sameness
that goes beyond featural overlap,
allowing us to apply the same operation to
features that were unseen during training.
Here, we identified translational symmetry as reflecting
the intrinsic structure of the problem, in terms of
inputs and outputs being sequences of binary digits.
