<jupyter_start><jupyter_text>Confidence intervals Intended Learning Objectives1. Calculate a confidence interval for a population mean2. Investigate how point estimates of the population mean vary from sample to sample3. Investigate how often confidence intervals calculated from a sample contain the population mean4. Be able to run and edit code from the statistics environment R in a Jupyter notebook  IntroductionIn this Jupyter notebook, we’ll simulate random samples from a population and calculate confidence intervals for the population mean.  We’ll see how the point estimates of the population mean vary from sample to sample and consider what proportion of the confidence intervals contain the true population mean.Each block of R code can be run by clicking 'Run' or by using the keyboard shortcut of Shift + Enter.  In fact, you can also edit the code before you run it.  Once you’ve worked through the notebook, try going back and changing some of the values used, like the size of each random sample or the number of samples taken.  What effect does this have?  If you want to change the population mean from 165 cm to some other value, how many changes to the code do you need to make in order to accurately calculate and present the results? Assume that we’re interested in the height in centimetres (cm) of women in the UK.  In particular we’d like to estimate what the average (i.e. mean) height of a woman in the UK is.  To do this, we might obtain the heights of a representative sample of women and calculate the mean height in our sample as an estimate of the population mean.We’ve seen that confidence intervals are a way for us to express how precise we believe our estimate to be.  To enable us to explore how confidence intervals vary from sample to sample, let’s assume for a moment that we know that heights of women in the UK follow a normal distribution with mean 165 cm and standard deviation 8.9 cm.  Of course, if we knew this we wouldn’t have to estimate what the mean height is, we’d already know.  But fixing what the true population mean is for a minute will allow us to explore how confidence intervals relate to the truth.Let’s simulate taking a random sample of size 50 from our assumed distribution, like we’ve obtained the heights of 50 women randomly selected from the population.<jupyter_code>sample.size <- 50
my.sample <- rnorm(n=sample.size, mean=165, sd=8.9)<jupyter_output><empty_output><jupyter_text>The mean height in our sample is<jupyter_code>x.bar <- mean(my.sample)
x.bar<jupyter_output><empty_output><jupyter_text>If we just had this one sample, this would be our estimate of what the mean height in the population is.  We could also estimate what the standard deviation of heights in the population is.<jupyter_code>s <- sd(my.sample)
s<jupyter_output><empty_output><jupyter_text>Our estimate of the population mean height is based on our sample and we would get another, almost certainly different, estimate from another sample.  How the mean would vary from sample to sample across all samples is known as the sampling distribution of the mean.  We can estimate the standard deviation of this distribution, known as the standard error of the mean, as<jupyter_code>se <- s / sqrt(sample.size)
se<jupyter_output><empty_output><jupyter_text>If the sampling distribution of the sample mean is a normal distribution, with standard deviation given by our estimate of the standard error, then we can calculate the lower and upper limits of a 95% confidence interval for the population mean as follows<jupyter_code>ci.lower <- x.bar - 1.96*se
ci.lower
ci.upper <- x.bar + 1.96*se
ci.upper<jupyter_output><empty_output><jupyter_text>We could write this in a sentence as follows<jupyter_code>paste("Our 95% confidence interval for the population mean is (",format(ci.lower,digits=5), ", ", format(ci.upper, digits=5),")", sep="")<jupyter_output><empty_output><jupyter_text>Now let’s take advantage of being in the rare position where we can take repeated samples from the population we’re interested in.We set the number of samples to take and create some objects to hold the results in for each sample: the sample mean, the lower and upper limits of the 95% confidence interval and whether or not the calculated confidence interval contains the true population mean<jupyter_code>n.samples <- 1000
x.bar <- vector(length=n.samples) 
cil <- vector(length=n.samples)
ciu <- vector(length=n.samples)
coverage <- vector(length=n.samples)<jupyter_output><empty_output><jupyter_text>We use a loop to perform the required calculation across the multiple samples, repeating this loop as many times as the number of samples we’re taking<jupyter_code>for(i in 1:n.samples){

    # generate the sample
    samp <- rnorm(sample.size, mean = 165, sd = 8.9)

    # sample mean
    x.bar[i] <- mean(samp)
    
    # sample standard deviation
    s <- sd(samp)

    # estimated standard error of the mean
    se <- s / sqrt(sample.size)
    
    # lower limit of the 95% confidence interval
    cil[i] <- x.bar[i] - 1.96*se

    # upper limit of the 95% confidence interval
    ciu[i] <- x.bar[i] + 1.96*se

    # does the CI contain the true population mean of 165 cm?: 0 for no, 1 for yes
    coverage[i] <- cil[i] < 165 & ciu[i] > 165
}<jupyter_output><empty_output><jupyter_text>Let’s look at a histogram of the sample means from all the samples we’ve just taken<jupyter_code>hist(x.bar, xlab="Height (cm)", main=paste("Estimated population means from ", n.samples ," samples", sep=""))

# add a vertical line to show the true population mean of 165 cm
abline(v=165, col="red", lwd=3)<jupyter_output><empty_output><jupyter_text>Now let’s draw all of the confidence intervals that we’ve just calculated, one for each sample<jupyter_code># set up the axes for the plot
plot(y=1:n.samples, x=x.bar, xlim=c(min(cil), max(ciu)), xlab="Height (cm)", ylab="sample #")

# add a blue line for any CI which does include the true population mean
segments(x0=cil[coverage], y0=which(coverage==1), x1=ciu[coverage], y1=which(coverage==1), col="blue")

# add a red line for any CI which does not include the true population mean
segments(x0=cil[!coverage], y0=which(coverage==0), x1=ciu[!coverage], y1=which(coverage==0), col="red")

# add a point to show the mean of each sample (in the middle of the CI)
points(y=1:n.samples, x=x.bar, pch=20, col="white")

# add a vertical line at the true population mean
abline(v=165, col="red", lwd=3)<jupyter_output><empty_output><jupyter_text>We’ve calculated the 95% confidence intervals using a technique which delivers a confidence interval containing the true population mean about 95% of the time.  How many of our confidence intervals contained the truth?  Are we close to that expected 95% coverage?<jupyter_code>cover.percent <- sum(coverage)/n.samples*100
cover.percent<jupyter_output><empty_output><jupyter_text>Let’s add our calculated coverage percentage to our plot<jupyter_code># set up the axes for the plot
plot(y=1:n.samples, x=x.bar, xlim=c(min(cil), max(ciu)), xlab="Height (cm)", ylab="sample #")

# add a blue line for any CI which does include the true population mean
segments(x0=cil[coverage], y0=which(coverage==1), x1=ciu[coverage], y1=which(coverage==1), col="blue")

# add a red line for any CI which does not include the true population mean
segments(x0=cil[!coverage], y0=which(coverage==0), x1=ciu[!coverage], y1=which(coverage==0), col="red")

# add a point to show the mean of each sample (in the middle of the CI)
points(y=1:n.samples, x=x.bar, pch=20, col="white")

# add a vertical line at the true population mean
abline(v=165, col="red", lwd=3)

mtext(paste(cover.percent, "% of these intervals contain 165"))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08214931542237149, 'fraction_numerical': 0.016404029966416947, 'mean_word_length': 5.04921875, 'pattern_counts': {'":': 0, '<': 70, '<?xml version=': 0, '>': 50, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '280321', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 2343, 'n_tokens_neox': 2204, 'n_words': 1028, 'path': 'Confidence Intervals.ipynb', 'repo_name': 'paulserafim/Jupyter-Notebook-', 'size': '7742'}