<jupyter_start><jupyter_text>Correlating reaction time with white noiseIn the cell below, we'll see how well RT correlates with random Gaussian / white noise.As expected, they only show a significant (p<=0.05) corellation ~5% of the time.<jupyter_code>surrogates = 1000

r_surr = np.zeros((surrogates, 1))
p_surr = np.zeros((surrogates, 1))

for i in range(surrogates):
    data = np.random.normal(0, 1, size=rt.size)
    r_surr[i], p_surr[i] = pearsonr(data, rt)

print(('median p-value = ' + str(np.median(p_surr))))
print(('percent of significant p-values = ' + str((np.count_nonzero(p_surr<=0.05)/surrogates)*100)))

sns.histplot(p_surr)
plt.show()<jupyter_output>median p-value = 0.49249527688340783
percent of significant p-values = 5.800000000000001<jupyter_text>Correlating reaction time with autocorrelated noiseNow, we'll do the exact same thing, but we're going to integrate the white noise. This induces a false dependence between successive data points (and is known as Brown noise).Perhaps unintuitively, fully ~50% of the time RT is significantly correlated to random Brown noise!<jupyter_code>surrogates = 1000

r_surr = np.zeros((surrogates, 1))
p_surr = np.zeros((surrogates, 1))

for i in range(surrogates):
    data = np.random.normal(0, 1, size=rt.size)
    data = np.cumsum(data)
    r_surr[i], p_surr[i] = pearsonr(data, rt)

print(('median p-value = ' + str(np.median(p_surr))))
print(('percent of significant p-values = ' + str((np.count_nonzero(p_surr<=0.05)/surrogates)*100)))

sns.histplot(p_surr)
plt.show()<jupyter_output>median p-value = 0.05310871907003229
percent of significant p-values = 48.5
{'fraction_non_alphanumeric': 0.12845728334357714, 'fraction_numerical': 0.054701905347264906, 'mean_word_length': 5.48605577689243, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '212371', 'license': 'Apache-2.0', 'n_tokens_mistral': 664, 'n_tokens_neox': 584, 'n_words': 154, 'path': 'Notebooks/COGS 280 - Autocorrelations.ipynb', 'repo_name': 'voytekresearch/ephys-class', 'size': '1627'}