<jupyter_start><jupyter_text>Pearson correlation coefficient$r_{x,y}=\frac{cov(x,y)}{\sqrt{S_{x}^2*S_{y}^2}}=\frac{cov(x,y)}{S_x*S_y}$$r \epsilon[-1;+1]$$cov(x, y) = \frac{\sum_{t=1}^{n}(x_t-\bar{x})(y_t-\bar{y})}{n}$$S^2=\frac{\sum_{t=1}^{n}(x_t-\bar{x})^2}{n}$<jupyter_code>import numpy as np
data = np.array([
    [1, 2, 3, 4], # x
    [8, 12, 17, 18] #y
])
n = len(data[0])
avg_x = np.average(data[0])
avg_y = np.average(data[1])
cov_diff = 0
for i in range(n):
    cov_diff = cov_diff + ((data[0, i] - avg_x) * (data[1, i] - avg_y))
cov = cov_diff / n
S2_x = 0 # variance
S2_y = 0 # variance

for i in range(n):
    S2_x = S2_x + np.power(data[0, i] - avg_x, 2)
    S2_y = S2_y + np.power(data[1, i] - avg_y, 2)

S2_x = S2_x / n
S2_y = S2_y / n
r = cov / np.sqrt(S2_x * S2_y)
print('Pearson correlation coefficient between X and Y is {}'.format(r))<jupyter_output>Pearson correlation coefficient between X and Y is 0.9725975251592748
{'fraction_non_alphanumeric': 0.22281449893390193, 'fraction_numerical': 0.06396588486140725, 'mean_word_length': 4.275280898876405, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '137329', 'license': 'MIT', 'n_tokens_mistral': 488, 'n_tokens_neox': 451, 'n_words': 93, 'path': 'pearson-correlation-coefficient.ipynb', 'repo_name': 'pilotpirxie/my-jupyter', 'size': '938'}