Finally, we investigate varying the function class of only the passive network by sharing the weights of the first \(k\)  (out of 5) layers of active and passive networks, while constraining the passive network to only update the remaining top \(5-k\)  layers, and using the
`representation' at layer \(k\)  acquired through active learning only. This reduces the `degrees of freedom' of the passive agent, which we hypothesize reduces its potential for divergence.
Indeed, Fig. REF  illustrates that passive performance correlates strongly with the number of tied layers, with the variant for which only the linear output layer is trained passively performing on par with the active agent. A similar result is obtained in the forked tandem setting, see Appendix Fig. REF .
This finding provides a strong indirect hint towards  REF: with only part of the network's layers being trained passively, much of its `generalization capacity' is shared between active and passive agents. States that are not aggregated by the shared bottom layers (only affected by active training) have to be `erroneously' aggregated by the remaining top layers of the network for over-generalization to occur. A more thorough investigation of this, exceeding the scope of this paper, may involve attempting to measure (over-)generalization more directly, e.g. via gradient interference [1]}.
