Single-phase deep learning in cortico-cortical networks
The error-backpropagation (backprop) algorithm remains the most common
solution to the credit assignment problem in artificial neural networks. In
neuroscience, it is unclear whether the brain could adopt a similar strategy to
correctly modify its synapses. Recent models have attempted to bridge this gap
while being consistent with a range of experimental observations. However,
these models are either unable to effectively backpropagate error signals
across multiple layers or require a multi-phase learning process, neither of
which are reminiscent of learning in the brain. Here, we introduce a new model,
bursting cortico-cortical networks (BurstCCN), which solves these issues by
integrating known properties of cortical networks namely bursting activity,
short-term plasticity (STP) and dendrite-targeting interneurons. BurstCCN
relies on burst multiplexing via connection-type-specific STP to propagate
backprop-like error signals within deep cortical networks. These error signals
are encoded at distal dendrites and induce burst-dependent plasticity as a
result of excitatory-inhibitory topdown inputs. First, we demonstrate that our
model can effectively backpropagate errors through multiple layers using a
single-phase learning process. Next, we show both empirically and analytically
that learning in our model approximates backprop-derived gradients. Finally, we
demonstrate that our model is capable of learning complex image classification
tasks (MNIST and CIFAR-10). Overall, our results suggest that cortical features
across sub-cellular, cellular, microcircuit and systems levels jointly underlie
single-phase efficient deep learning in the brain.