<jupyter_start><jupyter_text>The Box Plot  History of the Box Plot <NAME> (June 16, 1915 – July 26, 2000) ![<NAME>](John_Tukey.jpg) The box plot was invented by <NAME>.  <NAME> was a mathematician and professor in Princeton University.He introduced this type of visual data display in 1969 and later in his 1977 book "Exploratory Data Analysis".It was a way of visually representing the data.  As part of this visual display the data is summarised in an easily recognised visual way.We will show examples of the box plot below but here is a brief description of the box plot and the summary statistics that can be shown on the plot. The box plot, also know as a box and whisker plot, is a box with the end lines of the box at the 2nd and 3rd quartiles.  It shows the median as a line across the box.  The whiskers come out of the box with one extending to minimum and one extending to a maximum value.  You can also show outliers above or below the whiskers as the need arises.<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns<jupyter_output><empty_output><jupyter_text>Box Plot TerminologyBelow I have a picture of a box plot with labels on it. ![The Box Plot](Boxplot.png) Now I will say what each label is and we will look at a simple example below to explain the terminology in more detail.- **Max** is the Maximum Value in the range- **Whisker (from Max value)** = Upper 25% of values- **Q3** is the median of the third quartile- **Median** is the middle value of the range- **Box** is the main body of the range of observations, the middle 50%- **Q1** is the median of the Lower quartile- **Whisker (to the Minimum)** is the Lower 25% of values- **Min** is the Minimum Value in the range- **Q1** "the 25th percentile" - **Q2** is "the 50th percentile" - **Q3** is "the 75th percentile" - **Outliers** are not shown here but they are when there are values outside the range of the Min and Max values- **Interquartile Range (IQR)** is the middle 50% of observations, essentially the box of the boxplot Now I am going to show how powerful the box plot is by doing the calculations and explaining the values and then I will plot the same set of values in a box plot to show that a picture really does paint a 1000 words. Below we will make our own box plot as if we were doing the calculations ourselves. First of all I am using numpy.random to give me a random 20 numbers from 100.  I am using seed 1 so that the choice of 20 will not change.<jupyter_code>np.random.seed(seed=1) # seed 1 so that the output will not change
x = np.random.randint(100, size=15)  # 15 random number from 100
x<jupyter_output><empty_output><jupyter_text>- **Step 1** is to sort the numbers in numerical order.<jupyter_code>np.sort(x, axis=-1, kind='quicksort', order=None) # Sorting the array into numerical order<jupyter_output><empty_output><jupyter_text>- **step 2** is to find the median, in our example it is 37.  This number divides our array into 2 halves.  The median is the middle number.- **Step 3** is to divide the array into quarters.  We now find the median of the other 2 halves. We will now leave out the median in our example. Half 1 is 1,  5,  6,  9, 12, 16, 25, and half 2 is 50, 64, 71, 72, 75, 76, 79.  9 is the median of the first half and 72 is the median of the second half.We now have 3 points of interest, the median and the median points of the two halves, these three points divide our array into quarters, called "quartiles".  Q1 which is the lower quartile: 9.Q2 which is the median: 37.Q3 which is the upper quartile: 72.- Now we have the main body of the data which makes up the box of the box plot. The box of the box plot makes up the main bulk of the data between Q1 and Q3.- Now we can find the interquartile range (IQR) which is Q3 - Q1.  In our example the IQR is 72 - 9 = 63- The IQR is a useful number as it the range of the middle 50% of values and is therefore not influenced by low or high numbers.- The minimum value in our array is 1 and our maximum value is 79.<jupyter_code>df = pd.DataFrame(x) # reading our array into pandas
df.describe(percentiles=None, include=None, exclude=None) # Pandas to describe its attributes<jupyter_output><empty_output><jupyter_text>Please note above that the Q1 and Q3 values are slightly different to my figures.  This is because Pandas does not exclude the median when calculating the quartiles.  This gives us a chance to look at when there are an even number of observations and how you get the median in that case.  Looking again at Q1:  1,  5,  6,  9, 12, 16, 25, 37, now there is an even number of values so you have to take the average of the middle (9 + 12) / 2 = 10.5Now Q3: 37, 50, 64, 71, 72, 75, 76, 79, again an even number so we take (71 + 72) / 2 = 71.5<jupyter_code>data = x # calling our array X
plt.boxplot(data) # using x in a boxplot
plt.show() #  showing the plot<jupyter_output><empty_output><jupyter_text>Use of the box plot In looking at the use of the box plot I want to look at the temperatures over a 50 year period in the county that I live in. The data from the cso.ie at the weather station in Birr Co Offaly only goes to Sept 2009 so I will go back a corresponding 50 years to Sept 1959 giving me 600 lines of data.We can see if there is a change in the temperature observations over time.<jupyter_code>df = pd.read_csv("Birr Temperatures Degrees Box Plot.csv") # reading in the csv file of data
df
pd.PeriodIndex(start='1959-10', freq='M', periods=600) # adding a time series index to the data

df.index = pd.PeriodIndex(start='1959-10', freq='M', periods=600)
df
#df = pd.DataFrame(x) # reading our temperature data into pandas
df.describe(percentiles=None, include=None, exclude=None) # Pandas to describe its attributes<jupyter_output><empty_output><jupyter_text>We can see above the table containing the min, max, median at 50%, Q1 at 25% and Q3 at 75%.  Below this information is shown in a box plot. You can see the data summary of the data visualy in the plot.<jupyter_code>sns.set(style="whitegrid")
sns.set(rc={'figure.figsize':(15,10)})
sns.boxplot(x=None, y=None, hue=None, data=df, order=None, hue_order=None, orient=None, color=None, palette=None, saturation=0.75, width=0.8, dodge=True, fliersize=5, linewidth=None, whis=1.5, notch=False, ax=None)
# I want to split the data into 5 data frames of 10 years each
df_1 = df[:120] 
df_2 = df[120:240]
df_3 = df[240:360]
df_4 = df[360:480]
df_5 = df[480:600]
sns.set(style="whitegrid")
sns.set(rc={'figure.figsize':(15,10)})
sns.boxplot(x=None, y=None, hue=None, data=df_1, order=None, hue_order=None, orient=None, color=None, palette=None, saturation=0.75, width=0.8, dodge=True, fliersize=5, linewidth=None, whis=1.5, notch=False, ax=None)
sns.set(style="whitegrid")
sns.set(rc={'figure.figsize':(15,10)})
sns.boxplot(x=None, y=None, hue=None, data=df_2, order=None, hue_order=None, orient=None, color=None, palette=None, saturation=0.75, width=0.8, dodge=True, fliersize=5, linewidth=None, whis=1.5, notch=False, ax=None)
sns.set(style="whitegrid")
sns.set(rc={'figure.figsize':(15,10)})
sns.boxplot(x=None, y=None, hue=None, data=df_3, order=None, hue_order=None, orient=None, color=None, palette=None, saturation=0.75, width=0.8, dodge=True, fliersize=5, linewidth=None, whis=1.5, notch=False, ax=None)
sns.set(style="whitegrid")
sns.set(rc={'figure.figsize':(15,10)})
sns.boxplot(x=None, y=None, hue=None, data=df_4, order=None, hue_order=None, orient=None, color=None, palette=None, saturation=0.75, width=0.8, dodge=True, fliersize=5, linewidth=None, whis=1.5, notch=False, ax=None)
sns.set(style="whitegrid")
sns.set(rc={'figure.figsize':(15,10)})
sns.boxplot(x=None, y=None, hue=None, data=df_5, order=None, hue_order=None, orient=None, color=None, palette=None, saturation=0.75, width=0.8, dodge=True, fliersize=5, linewidth=None, whis=1.5, notch=False, ax=None)
df.boxplot(column='Birr Avg Max Temp (Deg C)', by=None, ax=None, fontsize=None, rot=0, grid=True, figsize=None, layout=None, return_type=None)
df.boxplot(column='Birr Avg Min Temp (Deg C)', by=None, ax=None, fontsize=None, rot=0, grid=True, figsize=None, layout=None, return_type=None)
df.boxplot(column='Birr Highest Temp (Deg C)', by=None, ax=None, fontsize=None, rot=0, grid=True, figsize=None, layout=None, return_type=None)
df.boxplot(column='Birr Lowest Temp (Deg C)', by=None, ax=None, fontsize=None, rot=0, grid=True, figsize=None, layout=None, return_type=None)
df.boxplot(column='Birr Mean Temp (Deg C)', by=None, ax=None, fontsize=None, rot=0, grid=True, figsize=None, layout=None, return_type=None)<jupyter_output><empty_output><jupyter_text>Box Plot Alternatives Swarmplot, very similar to the box plot but I do not think that it is as easy to read.<jupyter_code>sns.set(style="whitegrid")
sns.set(rc={'figure.figsize':(15,10)})
sns.boxplot(x=None, y=None, hue=None, data=df)
sns.swarmplot(x=None, y=None, hue=None, data=df, color='.25')<jupyter_output><empty_output><jupyter_text>The violin plot looks like it does not show outliers seperately and instead extends to them.  Visually it appears to be very similar in shape to the outline of the swarm of points in the swarm plot.<jupyter_code>sns.violinplot(x=None, y=None, hue=None, data=df, order=None, hue_order=None, bw='scott', cut=2, scale='area', scale_hue=True, gridsize=100, width=0.8, inner='box', split=False, dodge=True, orient=None, linewidth=None, color=None, palette=None, saturation=0.75, ax=None)<jupyter_output>C:\ProgramData\Anaconda3\lib\site-packages\scipy\stats\stats.py:1713: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  return np.add.reduce(sorted[indexer] * weights, axis=axis) / sumval<jupyter_text>The Boxenplot<jupyter_code>sns.boxenplot(x=None, y=None, hue=None, data=df, order=None, hue_order=None, orient=None, color=None, palette=None, saturation=0.75, width=0.8, dodge=True, k_depth='proportion', linewidth=None, scale='exponential', outlier_prop=None, ax=None)<jupyter_output><empty_output><jupyter_text>The Strip Plot, visually you can see straight away how tightly or not the points are grouped.  This plot does not give you the same high level data summary as a box plot from a quick glance.<jupyter_code>sns.stripplot(x=None, y=None, hue=None, data=df, order=None, hue_order=None, jitter=True, dodge=False, orient=None, color=None, palette=None, size=5, edgecolor='gray', linewidth=0, ax=None)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10783115540095892, 'fraction_numerical': 0.033562094575538215, 'mean_word_length': 5.40072202166065, 'pattern_counts': {'":': 0, '<': 48, '<?xml version=': 0, '>': 48, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '529596', 'license': 'Apache-2.0', 'n_tokens_mistral': 3757, 'n_tokens_neox': 3418, 'n_words': 1417, 'path': 'Box Plot.ipynb', 'repo_name': 'MichaelORegan/52446-Fundamentals-of-Data-Analysis-Project', 'size': '10653'}