<jupyter_start><jupyter_text>Project for Fundamentals of Data Analysis  An Investigation of the Box Plot  IntroductionThe box plot is a standardised way of displaying the distribution of data.  The box plot is a chart that graphically presents the five most important descriptive values for a data set - the minimum & maximum values, the median value and the first quartile and third quartile values.  A box plot is used in descriptive statistics as a method of graphically illustrating groups of numerical data through their quartiles.  The **first (1st) quartile** is the middle number which is between the smallest number of the data set (the **'minimum'** point) and the median (the middle value) of the data set.  The **second (2nd) quartile** is the **median** of the data set (the middle value).  The **third (3rd) quartile** is the middle value between the median and the highest value (the **'maximum'** point) of the data set.Box plots can also have lines which extend vertically from the boxes.  These lines are known as **whiskers**.  Whiskers are used to indicate the variability which is occuring outside the upper and lower quartiles.This is why the box plot is also known as the **box-and-whisker plot** and the **box-and-whisker diagram**.Box plots are used to display the variation in statistical samples.  The spaces between different parts of the boxes indicates the spread of the data.Box plots can be drawn either vertically or horizontally.  The diagram below illustrates a box plot.  The **whiskers** are the lines which extend from the box to the 'Maximum' and 'Minimum' points.**Outliers** are displayed above the maximum and minimum points.  These are 'surprisingly' high maximums or low minimums.  Turkey described the Outliers as either 3xIQR (likely range of variation) or more above the third quartile or 3xIQR below the first quartile.  There can also be **Suspected Outliers**.  These are slightly more central versions of outliers.  They can either be 1.5xIQR above the third quartile or 1.5xIQR below the first quartile. ![image.png](attachment:image.png)  HistoryIn 1969, mathematician <NAME> introduced the **Box and Whisker Plot**.  <NAME> (1915 - 2000) was an American mathematician.  He is credited with the development of the box plot and coining the term 'bit'.![image.png](attachment:image.png)Tukey introduced the box plot in his 1977 book, "Exploratory Data Analysis".  He used it as a visual diagram to illustrate the 'Five Number Summary' of any given data set.  A **Five Number Summary** includes the following data: * Minimum - the smallest number * First Quartile - the middle value between the 'minimum' and the 'median' * Median (Second Quartile) * Third Quartile - the middle value between the 'median' and the 'maximum' value * Maximum - the highest value When graphing the five number summary only the horizontal axis displays values.  Within the quadrant, a vertical line is placed above each of the summary numbers.  A box is drawn around the middle three lines (first quartile, median and the third quartile).  Then two lines are drawn from the box's edge to the two endpoints which are the minimum and maximum points.Since Tukey introduced the box plot there have been several variations introduced.  These include the variable width box plot and the notched box plots.  **Variable width box plots** are used to illustrate the size of each group whose data is being plotted, as it makes the box width proportionate to the size of the group.  Often the box width is made proportionate to the square root of the size of the group.  **Notched box plots** apply a 'notch' or a narrowing of the box which is around the median.  The notches are used to illustrate the difference of the medians.  For example, if the notches of two boxes do not overlap then this illustrates the statistical difference between the medians. Box plots are used to show the shape of a distribution, its central value and its variability.Examples of a Variable Width Box Plot and a Notch Box Plot can be seen below<jupyter_code># Example of a Variable Width Box Plot

# Import matplotlilb, numpy and panda
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

# Set the seed
np.random.seed(0)

# Set size, labels and font
data = np.random.lognormal(size=(39, 4), mean=1.5, sigma=1.75)
labels = list('ABCD')
fs = 12 

# Plot
plt.boxplot(data, patch_artist=True, labels=labels, showfliers=False, widths=(1.5, 0.1, 1.0, 0.5))

# Display the plot
plt.show()
# Set the size, labels and font
data = np.random.lognormal(size=(39, 4), mean=1.5, sigma=1.75)
labels = list('ABCD')
fs = 12  

# Plot now includes 'notch'
plt.boxplot(data, notch='True', patch_artist=True, labels=labels, showfliers=False, widths=(1.0, 0.1, 1.0, 0.5))
                                                                                        
# Display the plot
plt.show()<jupyter_output><empty_output><jupyter_text>Use of the BoxplotThe below code is used to generate a random data set.  This data set is then used to generate four different types of plots:   1  Box Plots    2  Histogram    3  Scatter Plot    4  Line Plot  The means of each group are plotted by each type of plot.  These illustrate how the same data set looks completely different on each type of plot.  The box plot automatically plots the median (the red line in each box).  The box plot is the only graph, of the four, that accurately illustrates the variability of the maximum and minimum points, this is shown in the first box plot on the graph which shows the whiskers stretching down the minimum point (the lowest value on the dataset) and stretching up to the maximum point (the highest value).  Outliers are illustrated as small circles above and/or below the whiskers.In a box plot the central rectangle goes from the first quartile to the third quartile.  A segment inside the rectangle shows the median and "whiskers" above and below the box.  These whiskers show the locations of the minimum and the maximum points.<jupyter_code># Import matplotlilb, numpy and panda
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
# Import rc from matplotlib to create the dataset

from matplotlib import rc

# Sets the font to DejaVu Sans and the font size to 18
rc('font',**{'family':'sans-serif','sans-serif':['DejaVu Sans'],'size':18})

rc('mathtext',**{'default':'regular'})

%matplotlib inline
# to create the plots (format the axis for the four graphs)

fig, ax = plt.subplots()
fig, ax = plt.subplots(nrows=2, ncols=2, figsize=(8,6))
plt.close(); # to hide the plots

# Ax is a 2x2 numpy.ndarray of matplotlib.axes.
type(ax), ax.shape
plt.close(); # to hide the plots
# The mean and standard deviation for the groups
data_specs = [(4, 4), (9, 3), (8, 4.5), (12, 0.5), (7.5, 0.1)]

# Create a data frame with random data
data = [np.random.normal(mu, sigma, 10) for mu, sigma in data_specs]
data = pd.DataFrame(data).T
data.columns = ['Group_%s' % n for n in range(1,6)]
data
fig, ax = plt.subplots(nrows=2, ncols=2, figsize=(10,8))

# Finds the mean of each group
means = data.mean()

# Create x coordinates
x = np.arange(len(means))

# Creates a boxplot of the data values in each group
ax[0][0].boxplot(data.values, positions=x)

# Create a histogram of the means of each group
ax[0][1].bar(x, means)

# Create a scatter plot of the means of each group
ax[1][0].scatter(x, means)

# Create a line plot of the means of each group
ax[1][1].plot(x, means);<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07400986798239766, 'fraction_numerical': 0.014401920256034139, 'mean_word_length': 4.237430167597766, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '363714', 'license': 'MIT', 'n_tokens_mistral': 2182, 'n_tokens_neox': 2022, 'n_words': 1102, 'path': 'Box-Plot.ipynb', 'repo_name': 'colettegallagher/box-plot', 'size': '7514'}