Conceptually speaking, aren't artificial neural networks just highly distributed, lossy compression schemes?They're certainly efficient at compressing images.And aren't brains (at least, the neocortex) just compartmentalized, highly distributed, lossy databases?If so, what salient features in RNNs and CNNs are necessary in any given lossy compression scheme in order to extract the semantic relations that they do? Is it just a matter of having a large number of dimensions/variables? Could some kind of lossy Bloom filter be re-purposed for the kinds of problems ANNs are applied to?
Are ANNs just highly distributed lossy compression schemes?
neural networks
Auto-encoders, a family of ANNs, are trained with exactly compression in mind. So definitely some ANNs are compressors.Also, in general, ANNs learn the best concepts to minimize fitness error. I would say that that means, both in classification and regression, to 1) differentiate between various inputs and 2) output the proper value for each input. Point 1), in particular, means having as many distinct net activation configurations as necessary to (at least) tell apart inputs needing (significantly, not in a statistical sense, just a qualitative sense) different outputs.I am inclined to think that if the input is complex enough, you would call what happens compression.