<jupyter_start><jupyter_text>Time series analysisNext, we'll do some analysis on the generated time serie(s) using available functions in `scipy` and `numpy`. moving average<jupyter_code>def simple_moving_average(y, N):
    y_padded = np.r_[np.repeat(y[0], N), y]
    cs = np.cumsum(y_padded)
    return (cs[N:] - cs[:-N]) / N

def centered_moving_average(y, N):
    y_padded = np.r_[np.repeat(y[0], (N+1)//2), y, np.repeat(y[-1], N//2)]
    cs = np.cumsum(y_padded)
    return (cs[N:] - cs[:-N]) / N
y_sma_10 = simple_moving_average(y, 1000)
y_cma_10 = centered_moving_average(y, 1000)

fig1 = plt.subplot(221)
fig1.plot(t, y)
fig1.plot(t, y_sma_10)
fig1.set_ylabel('N')
fig1.set_title('simple moving average')

fig2 = plt.subplot(223)
fig2.plot(t, y - y_sma_10)
fig2.set_xlabel('t')
fig2.set_ylabel('$\Delta N_{1000}$')

fig3 = plt.subplot(222)
fig3.plot(t, y, label='$N(t)$')
fig3.plot(t, y_cma_10, label='$N_{1000}(t)$')
fig3.set_title('centered moving average')
fig3.legend(loc='upper right', shadow=True)

fig4 = plt.subplot(224)
fig4.plot(t, y - y_cma_10)
fig4.set_xlabel('t')

plt.savefig('figures/moving_average.svg')
plt.show()<jupyter_output><empty_output><jupyter_text>During the lecture we saw how to compute the standard deviation on a moving window like this<jupyter_code># compute moving std
residual = y - y_cma_10
n_window = 2000

idx = np.arange(n_window)[None,:] + np.arange(len(t) - n_window)[:, None]
stds1 = residual[idx].std(axis=1)<jupyter_output><empty_output><jupyter_text>On the way back in the train I realized there is a much simpler way! The variance is defined as as$$\sigma^2 := \langle (x - \langle x \rangle)^2 \rangle.$$And we can use the identity$$\sigma = \sqrt{\langle x^2 \rangle - \langle x \rangle^2}.$$<jupyter_code>residual = y - centered_moving_average(y, n_window)
stds2 = np.sqrt(centered_moving_average(residual**2, n_window) - centered_moving_average(residual, n_window)**2)
fig = plt.subplot(111)
fig.plot(t[n_window:-n_window], stds2[n_window:-n_window])
fig.set_xlabel('t')
fig.set_ylabel('$\sigma_{1000}$')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16666666666666666, 'fraction_numerical': 0.040952380952380955, 'mean_word_length': 5.691082802547771, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '264110', 'license': 'Apache-2.0', 'n_tokens_mistral': 924, 'n_tokens_neox': 851, 'n_words': 177, 'path': 'Data analysis.ipynb', 'repo_name': 'jhidding/stability', 'size': '2100'}