Do Neural Networks Compress Manifolds Optimally?
Artificial Neural-Network-based (ANN-based) lossy compressors have recently
obtained striking results on several sources. Their success may be ascribed to
an ability to identify the structure of low-dimensional manifolds in
high-dimensional ambient spaces. Indeed, prior work has shown that ANN-based
compressors can achieve the optimal entropy-distortion curve for some such
sources. In contrast, we determine the optimal entropy-distortion tradeoffs for
two low-dimensional manifolds with circular structure and show that
state-of-the-art ANN-based compressors fail to optimally compress the sources,
especially at high rates.