Discretization can be used to bottleneck communication in modular inference models, for example in multi-agent reinforcement learning.
Modular inference combines outputs across modules into new module inputs while restricting the number of modules that communicate [1]}.
To make up for the loss in expressivity from being restricted to few communicative modules in each timestep, the model is forced to learn to perform inference in steps of specialized skill as opposed to applying all skills at once, which specializes the modules by definition since few are active in each step
[2]}, [1]}, [4]}, [5]}.
The communication bottleneck is tightened by switching from continuous to discrete representations [6]}.
Beyond improvements in generalization error from decreasing capacity, it has been hypothesized that the inductive bias of bottlenecked communication between modules improves generalization by 1) reflecting the true causal structure of data generated by sparse interactions between few variables, 2) increasing robustness when modules are recombined in novel ways, compared to unspecialized modules with all-to-all communication, 3) improving sample efficiency since specialized modules require fewer training points to learn
[1]}, [4]}.
