Essentially the information needs to be compressed
in some way to be stored in fewer variables. Formally this
can be expressed by mapping the high-dimensional space through
the bottleneck, which we hope results in recovering the low dimensional
concepts that created the high-dimensional representation in the
first place. In any case it means that typical features are created, with
a similar intuition to efficient codes in compression:
If a simple feature occurs often, giving it a representation
will reduce the loss more than representing a less common feature.
In fact, compression algorithms can be seen as a kind of feature
creation[1]}.
