Using basic Backpropagation and a Sigmoid activation function, solving for $\cfrac{\partial E}{\partial W_i}$ for all $W_i$ is rather straightforward. However, when using time series data, how do you approach backpropagation? My intuition tells me that you just do:$$(1) \sum_{t=1}^n \cfrac{\partial  E_t}{\partial W_i} = \sum_{t=1}^n \cfrac{\partial  E_t}{\partial o_{i_t}} \cfrac{\partial o_{i_t}}{\partial W_i}$$where $o_{i_t}$ is the output node at time $t$, and $W_i$ is the weight feeding into that node from the previous node, $x_i$, across all times $t = 1,..., n$. That would mean, if $\delta_{o_i} = \cfrac{\partial E}{\partial \sigma(o_i)} \cfrac{\partial \sigma (o_i)}{\partial o_i}$, where $\sigma (x) = \cfrac{1}{1+e^{-x}}$then the appropriate representation of $\delta_{o_i}$ within the time series should be:$$\sum_{t=1}^n \delta_{o_i} = \sum_{t=1}^n \cfrac{\partial E_t}{\partial \sigma(o_{i,t})} \frac{\partial \sigma (o_{i,t})}{\partial o_{i,t}}$$or rewritten, $$\sum_{t=1}^n \delta_{o_i,t} \frac{\partial o_{i,t}}{\partial W_i} = \sum_{t=1}^n (Out_t-Target_t)(Out_t)(1-Out_t)(x_{i,t})$$This requires me to calculate $n$ backpropagation problems per iteration. Is this the proper way to do it? It seems that if I use this method and a particular node adds random error across the time series (say, the mean of $\cfrac{\partial  E_t}{\partial W_i}$ for all $t$ is roughly $0$), then the backpropagation algorithm will not adjust that weight since $$\sum_{t=1}^n \cfrac{\partial  E_t}{\partial W_i}$$ will be close to 0.Are there any brief summaries of how to approach backpropagation across a time series? Or should I just train the network using each observation in the time series as an independent training point?
Backpropagation in a neural network with time series data?
neural network;backpropagation