I've been trying to implement a simple convolutional neural network. But I've been stuck at this problem for over a week. To be specific, assume there are 3 layers in a convolutional pass, marked as l-1, l, l+1 -layer respectively. And the l-1 layer is the input layer with shape (num_channels, img_height, img_width),the l layer is the convolutional layer with kernel shape (num_kernels, num_channels, filter_height, filter_width), and the l+1 layer is the pooling layer with pooling size (pool_height, pool_width).  Note that in my simple implementation, the filters are 3-dimensional, and they have the same depth as with the input, i.e., they have the same number of channels as that of the input.On this setting, I encountered a problem in the back-propagation phase.I've studied many tutorials in neural networks, however, it seems that all of them stop with back-propagating the error to l layer with the formular:\begin{align}   \delta_k^{(l)} = \text{upsample}\left((W_k^{(l)})^T \delta_k^{(l+1)}\right) \bullet f'(z_k^{(l)})   \end{align}The question is, how should I back-propagate error from the l layer to l-1 layer, that is, calculate $\delta_k^{(l-1)}$, in order to chain several conv-pooling pass together?In my implementation, there is a class called ConvPoolLayer which has a feedforward method and a backprop method. The method backprop needs to propagate the error from output to input. When I chain instants of class ConvPoolLayer together, I just need train their feedforward methods and backprop methods together respectively. 
how to propagate error from convolutional layer to previous layer
machine learning;neural network;deep learning;convnet;backpropagation