[1]} correctly identifies
training independence as a factor limiting
the ability of standard connectionist architectures
to generalise outside their training space.
That is, without additional constraints, the backpropagation
algorithm adjusts the weights associated
with each input feature independently.
This flexibility makes them one of the most effective
methods of machine learning when training data is plentiful
and the test data is drawn from an identical distribution.
However, in the identity [2]} and
rule [3]} learning tasks,
this results in these models failing to generalise
to the unseen features.
