<jupyter_start><jupyter_text>What is standard deviation or variance?- We measured number of rainy days at fall in two different cities in last 5 years:<jupyter_code>City_A = [29, 28, 32, 35 ,36]

import numpy as np

print(np.mean(City_A))
City_B = [45, 18, 30, 12, 55]
print(np.mean(City_B))<jupyter_output>32.0<jupyter_text>Question: Which city has greater variance?- In other words, which city has more variation?<jupyter_code>print(np.std(City_A))
print(np.std(City_B))<jupyter_output>16.112107248898266<jupyter_text>Most of the values (rainy days) in Cities A, B are between mean - std and mean + std<jupyter_code>[32 - np.std(City_A), 32 + np.std(City_A)]
[32 - np.std(City_B), 32 + np.std(City_B)]
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt

df = pd.DataFrame({'Rainy':[29,28,32,35,36,12,18,30,45,55, 32,32,32,32,32], 'City':['City_A']*5 + ['City_B']*5 + ['City_C']*5})

sns.lineplot(y='Rainy', x='City', data=df)


plt.show()<jupyter_output><empty_output><jupyter_text>Two Normal distributions with the same mean and different standard deviation<jupyter_code>import numpy as np
import seaborn as sns

a = np.random.normal(60, 10, 100000)
sns.distplot(a, hist=True, kde=True, bins=40)

b = np.random.normal(60, 20, 100000)
sns.distplot(b, hist=True, kde=True, bins=40)<jupyter_output><empty_output><jupyter_text>About 68% of the values in a are in [50, 70] rangeAbout 68% of the values in b are in [40, 80] range  How to compute CDF<jupyter_code>cdf_fare = df['Fare'].value_counts().sort_index().cumsum()/len(df['Fare'])
cdf_fare.plot()
def calculate_cdf(x, data):
    return np.sum(x < data)


cdf_a = [calculate_cdf(a, r)/len(a) for r in range(int(np.min(a)), int(np.max(a)))]

cdf_b = [calculate_cdf(b, r)/len(b) for r in range(int(np.min(b)), int(np.max(b)))]

import matplotlib.pyplot as plt

plt.plot(range(int(np.min(a)), int(np.max(a))), cdf_a)
plt.plot(range(int(np.min(b)), int(np.max(b))), cdf_b)<jupyter_output><empty_output><jupyter_text>Another way we can compute the CDF of a and b- To compute the CDF of a, we can sort element in a. But the size of cdf_a would be unnecessary large compared to `range(int(np.min(a)), int(np.max(a)))`<jupyter_code>def calculate_cdf(x, data):
    return np.sum(x < data)


cdf_a = [calculate_cdf(a, r)/len(a) for r in sorted(a)]
plt.plot(sorted(a), cdf_a)<jupyter_output><empty_output><jupyter_text>Z-Distribution - When the samples of our numerical array are Normal with arbitrary mean and std- If scale each element by subtracting from the mean and divide over std, then the new array list would be a Normal distribution with zero mean, std 1 - Z-distribution is another name for standard Normal distribution  Activity: Show that<jupyter_code>c = (b - 60)/20
sns.distplot(c, hist=True, kde=True, bins=40)
d = (a - 60)/10
sns.distplot(d, hist=True, kde=True, bins=40)<jupyter_output><empty_output><jupyter_text>Pair Programming- From now till the end of Term 3, I want to change solo programming routine to formal pair programming- Read this blog to get familiar with formal pair programming and how it works: https://builttoadapt.io/a-recipe-for-successful-pair-programming-b5eec6611e7- One programming would be Driver and the other Navigator: https://gist.github.com/jordanpoulton/607a8854673d9f22c696<jupyter_code>import random


ls = ['Jasmine', 'Jayce', 'Cherish', 'Colleen', 'Conner', 'Dacio', 'Egon', 'Eric', 'Ikey', 'Jason', 'Jammie', 'Lucia', 'Nathan', 'Reymond', 'Samuel', 'Sukhrob', 'Faith', 'Vincenzo']
pair = list()
a_ls = ls
for i in a_ls:
    if len(a_ls) >= 2:
        a = random.sample(a_ls, 2)
        pair.append(a)
        a_ls = list(set(a_ls) - set(a))
    else:
        break
    print(i)


print(pair)
# print(ls)
print(a_ls)
print(len(ls))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14417989417989419, 'fraction_numerical': 0.04126984126984127, 'mean_word_length': 5.452218430034129, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 32, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '95924', 'license': 'MIT', 'n_tokens_mistral': 1591, 'n_tokens_neox': 1443, 'n_words': 396, 'path': 'Notebooks/variance_pdf_cdf.ipynb', 'repo_name': 'sidneyarcidiacono/DS-1.1-Data-Analysis', 'size': '3780'}