<jupyter_start><jupyter_text>Law of Large Numbers (LLN)  Definition As the number of experiment repetitions increases, the average of the sample means better approximates the population mean. ![38E44E4F-E353-4894-A62F-E8962F7A5E0F.jpeg](data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAkACQAAD/4QCARXhpZgAATU0AKgAAAAgABQESAAMAAAABAAEAAAEaAAUAAAABAAAASgEbAAUAAAABAAAAUgEoAAMAAAABAAIAAIdpAAQAAAABAAAAWgAAAAAAAACQAAAAAQAAAJAAAAABAAKgAgAEAAAAAQAABDWgAwAEAAAAAQAAAOkAAAAA/+0AOFBob3Rvc2hvcCAzLjAAOEJJTQQEAAAAAAAAOEJJTQQlAAAAAAAQ1B2M2Y8AsgTpgAmY7PhCfv/iAjRJQ0NfUFJPRklMRQABAQAAAiRhcHBsBAAAAG1udHJSR0IgWFlaIAfhAAcABwANABYAIGFjc3BBUFBMAAAAAEFQUEwAAAAAAAAAAAAAAAAAAAAAAAD21gABAAAAANMtYXBwbMoalYIlfxBNOJkT1dHqFYIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACmRlc2MAAAD8AAAAZWNwcnQAAAFkAAAAI3d0cHQAAAGIAAAAFHJYWVoAAAGcAAAAFGdYWVoAAAGwAAAAFGJYWVoAAAHEAAAAFHJUUkMAAAHYAAAAIGNoYWQAAAH4AAAALGJUUkMAAAHYAAAAIGdUUkMAAAHYAAAAIGRlc2MAAAAAAAAAC0Rpc3BsYXkgUDMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAdGV4dAAAAABDb3B5cmlnaHQgQXBwbGUgSW5jLiwgMjAxNwAAWFlaIAAAAAAAAPNRAAEAAAABFsxYWVogAAAAAAAAg98AAD2/////u1hZWiAAAAAAAABKvwAAsTcAAAq5WFlaIAAAAAAAACg4AAARCwAAyLlwYX<KEY>AAAA<KEY>AFABQAUAFABQAUAFAB<KEY>AFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFABQAUAFAB<KEY>AKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKACgAoAKAP//Z) µ = population mean (true value)x(bar)n = sample mean (the average of the sample population)As the limit nears infinity (the sample population increases), the probability of the difference between the sample mean and the population mean (abs value) being greater than an arbitrarily small number (epsilon) nears 0. **Essentially, one sample or one experiment is unlikely to provide a good estimate of the true population mean.**Sampling many times can provide an accurate estimation of the true population mean.  Generate a population of numbers<jupyter_code>import matplotlib.pyplot as plt
%matplotlib inline

import pandas as pd
import numpy as np

import seaborn as sns
sns.set_style("whitegrid")
# Simulation parameters

population_size = 2.3e5

# Generate the population of numbers

population = 1/ np.logspace(np.log10(.001),np.log10(10),int(population_size))

skip = int(1e3)

plt.plot(population[::skip], ".")
plt.xlabel("Sample")
plt.ylabel("Data")
# Shuffle data

np.random.shuffle(population)

plt.plot(population[::skip], ".")
plt.xlabel("Sample")
plt.ylabel("Data")
# True mean

true_mean = np.mean(population)<jupyter_output><empty_output><jupyter_text>Monte Carlo sampling<jupyter_code># Simulation parameters

sample_size = 50

sample_number = 500<jupyter_output><empty_output><jupyter_text>Draw and average {sample_size} data points at random from the population {sample_number} times in total.<jupyter_code>sample_means = np.zeros(sample_number)

for exp in range(sample_number):
  randsample = np.random.choice(population, size = sample_size)
  sample_means[exp] = np.mean(randsample)
plt.plot(sample_means, "ko", label="Sample means")
plt.plot([0,sample_number], [true_mean, true_mean], "r", linewidth=5, label="True mean")
plt.legend()
plt.xlabel("Sample number")
plt.ylabel("Mean value")<jupyter_output><empty_output><jupyter_text>Cumulative averaging<jupyter_code>def cum_avg(arr):
  cum_sum = np.cumsum(arr,axis=0)
  for i in range(cum_sum.shape[0]):
    if i == 0:
      continue
    cum_sum[i] = cum_sum[i]/(i+1)
  return cum_sum
cumulative_mean = cum_avg(sample_means)

plt.plot(cumulative_mean, "blue", label="Cumulative mean")
plt.plot([0,sample_number], [true_mean, true_mean], "r", linewidth=5, label="True mean")
plt.legend()
plt.xlabel("Sample number")
plt.ylabel("Cumulative mean")
meta_samples = 100

cumavgs = np.zeros((meta_samples, sample_number))
allsamplemeans = np.zeros((meta_samples, sample_number))

for meta in range(meta_samples):
  for exp in range(sample_number):
    randsample = np.random.choice(population, size = sample_size)
    sample_means[exp] = np.mean(randsample)
  temp = np.cumsum(sample_means)/np.arange(1,sample_number+1)
  cumavgs[meta,:] = (temp-true_mean)**2 
  allsamplemeans[meta,:] = sample_means
plt.plot(cumavgs.T)
plt.ylim([-20, 500])
plt.legend()
plt.xlabel("Sample number")
plt.show()<jupyter_output>No handles with labels found to put in legend.<jupyter_text>Central limit theorem The distribution of sample means ~ Gaussian<jupyter_code>plt.hist(sample_means, bins="fd")

plt.xlabel("Sample mean")
plt.ylabel("Count")

nbins = 40
for i in range(meta_samples):
  y,x = np.histogram(allsamplemeans[i,:],bins=nbins)
  x = (x[1:]+x[:-1])/2
  plt.plot(x,y, color="grey", alpha=0.4)

plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09229249011857707, 'fraction_numerical': 0.021146245059288537, 'mean_word_length': 9.307535641547862, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 11, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '363750', 'license': 'MIT', 'n_tokens_mistral': 2212, 'n_tokens_neox': 2026, 'n_words': 303, 'path': 'stats/law-of-large-numbers/lln.ipynb', 'repo_name': 'the-cryptozoologist/data-science', 'size': '65347'}