The connections are inspired by how feed-forward connections in the mammalian brain are highly overlapping and convolutional ([1]}, [2]}) and hierarchical ([3]}), being the inspiration for Convolutional Neural Networks (CNNs)[4]}. However, unlike CNN's, the kernels do not share weights, in an effort to remain biologically realistic, as well as to investigate if the non-shared quality of weights is an important factor in a spiking network. The connection schema is loosely based on a mix of [5]} and modern CNN architectures. Each neuron has feed-forward connections to a small square block of neurons in the post-layer, exactly as in a CNN. This is done intentionally because the network is not learning a kernel using backpropagation, and needs as much variation as possible to aid in the STDP process. In addition, in attempting to reverse-engineer the fundamental principles of spiking, it is biologically implausible that neurons communicate some sort of shared weight across the cortex; weight updates are local by nature. The weights are initialized with a uniform random distribution, and given an equally-high starting weight that allows spikes to occur so that STDP can begin reshaping the weights, consistent with theories that state that intrinsically high starting activation is a major part of neuronal group formation ([6]}). Excitatory weights are capped at +7 and inhibitory at -30, and STDP is only performed on feed-forward connections.
In addition to the feed-forward connection, each neuron also connects horizontally to a square of neurons in the same layer. This provides a degree of recurrence to the system, implementing a version of "re-entrant" connections as described by Edelman ([7]}), although more long-distance reentry is left for future work. We refer to the system as "semi-recurrent" and not fully recurrent because it does not contain backward connections to previous layers.
