<jupyter_start><jupyter_text>Central Limit Theorem The central limit theorem states that the sum of a number of independent and identically distributed (i.i.d.) random variables with finite non-zero variances will tend to a normal distribution as the number of variables grows.  The mean for both the populationa and the samples is the same and the standard deviation of the sample and the standard deviation of the population are related by:$\LARGE  SE = \frac{\sigma}{\sqrt{2}{n}}$  This is known as the **standard error**.  Obviously, as n increases the standard error is smaller.<jupyter_code>import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from itertools import combinations_with_replacement<jupyter_output><empty_output><jupyter_text>Example : Uniform Distribution<jupyter_code>size = 1000
population = np.random.randint(low=1, high=7, size=size)
print(f'Population = {population[:20]} ...')
print(f'Sigma = {np.mean(population)} STD = {np.std(population):.2f}')
bins = len(set(population))
sns.distplot(population, bins=bins)

n = 5  # Sample size
num_of_samples = 10000
samples = [ np.random.choice(population,n) for x in range(num_of_samples) ]
ev_samples = [np.average(x) for x in samples]
print(f'Samples = {[tuple(x) for x in samples][:5]} ...')
print(f'Sigma_ = {np.mean(ev_samples):.2f} STD = {np.std(ev_samples):.2f}')
bins = len(set(ev_samples))
print(f'Average of samples : {np.around(ev_samples[:5],decimals=2)}...    bins : {bins}')
sns.distplot(ev_samples, bins=bins)<jupyter_output>Population = [6 6 2 5 1 3 4 1 2 3 2 6 6 1 5 5 1 4 1 2] ...
Sigma = 3.413 STD = 1.68
Samples = [(5, 2, 4, 1, 6), (1, 4, 4, 2, 2), (3, 1, 4, 2, 5), (5, 5, 3, 5, 2), (6, 2, 2, 6, 2)] ...
Sigma_ = 3.41 STD = 0.75
Average of samples : [ 3.6  2.6  3.   4.   3.6]...    bins : 25
{'fraction_non_alphanumeric': 0.14055555555555554, 'fraction_numerical': 0.049444444444444444, 'mean_word_length': 4.541538461538462, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '353167', 'license': 'MIT', 'n_tokens_mistral': 716, 'n_tokens_neox': 622, 'n_words': 219, 'path': 'dsref/notebooks/Statistics.ipynb', 'repo_name': 'cesell/Data_Science_Reference', 'size': '1800'}