One of the fundamental challenges in the supervised learning of SNNs using gradient methods is the non-differentiability of the discontinuous binary spike function. Various approaches have been developed and successfully overcome this difficulty [1]}, [2]}, [3]}. Most of those used in the recent research can be divided into two categories depending on a state quantity, through which the error signals are backpropagated: the rate-based, and the timing-based method.
First, the rate-based method calculates the derivatives between time-evolving functions such as membrane potentials and spike functions, which leads to excellent results in a variety of tasks [4]}, [5]}. However, to achieve the functional derivatives, the rate-based method requires the time discretization of each function and the approximation of the gradient computation by replacing discontinuous spike gradients with continuous surrogate gradients (SG) during error backpropagation. In addition, there are concerns about how accurately the transfer of information is approximated due to errors introduced by these techniques of the rate-based method. Note that the use of SGs, which are continuous real-valued functions, can be interpreted as approximating the spike rate of each neuron with an activation function, suggesting that the network is not efficiently dealing with exact spike timing. Therefore, it is somewhat questionable whether it is appropriate as a temporal-coding scheme.
