Predictive coding, within theoretical neuroscience, and variational autoencoders, within machine learning, both involve latent Gaussian models and variational inference. While these areas share a common origin, they have evolved largely independently. We outline connections and contrasts between these areas, using their relationships to identify new parallels between machine learning and neuroscience. We then discuss specific frontiers at this intersection: backpropagation, normalizing flows, and attention, with mutual benefits for both fields. Perception has been conventionally formulated as hierarchical feature detection [52] , similar to discriminative deep networks [34] . In contrast, predictive coding [48, 14] and variational autoencoders (VAEs) [31, 51] frame perception as a generative process, modeling data observations to learn and infer aspects of the external environment. Specifically, both areas model observations, x, using latent variables, z, through a probabilistic model, p θ (x, z) = p θ (x|z)p θ (z). Both areas also use variational inference, introducing an approximate posterior, q(z|x), to infer z and learn the model parameters, θ. These similarities are the result of a common origin, with Mumford [45] , Dayan et al. [9] , and others [46] formalizing earlier ideas [59, 38] . However, since their inception, these areas have developed largely independently. We explore their relationships (see also [58, 37] ) and highlight opportunities for the transfer of ideas. In identifying these ties, we hope to strengthen this promising, close connection between neuroscience and machine learning, prompting further investigation. We have identified commonalities between predictive coding and VAEs, discussing new frontiers resulting from this perspective. Reuniting these areas may strengthen the connection between neuroscience and machine learning. Further refining this connection could lead to mutual benefits: neuroscience can offer inspiration for investigation in machine learning, and machine learning can evaluate ideas on real-world datasets and environments. Indeed, despite some push back [17] , if predictive coding and related theories [18] are to become validated descriptions of the brain and overcome their apparent generality, they will likely require the computational tools and ideas of modern machine learning to pin down and empirically compare design choices.
connections between predictive coding and VAEs + new frontiers
{"src": "scitldr-AIC_train", "paper_id": "SyeumQYUUH"}