<jupyter_start><jupyter_text>Fundamentals of Data Analysis Project 2018 <NAME>.Dip in Data Analytics Problem StatementThe following assignment concerns the boxplot function. There are four distinct tasks which are carried out below:* Summarise the history of the box plot and situations where it is used.* Demonstrate the use of the box plot using data of my choice.* Explain any relevant terminology such as the terms quartile and percentile.* Compare the box plot to alternatives.  The Box-PlotIt was <NAME> (a Statiscian at Princeton University) who invented the box and whiskers plot as part of his toolkit for exploratory data analysis in 1970, but it was not until 1977 when his book was formally published that it became more widely known. The boxplot is a compact distributional summary, that shows less detail than a histogram or kernel density, and also takes up less space than a histogram. Boxplots use robust summary statistics that are always located at actual data points, quick to compute, and have no tuning parameters. Box plots can be extremely useful for comparing distributions across different groups.A boxplot which can also be referred to as a box and whiskers plot is a plot which can be used to display the spread of a set of data. There are a number of ways which we can measure this spread of data which include the mean of the set of data and also interquartile range of this data. The measures which are used for the centre of the box plot include mean and median of the data set.The box and whiskers data chart displays to us how the data set is spread out. There are usually five key measures which are used in the chart these include the minimum which is always the smallest number that is in the data set – we can see that this is displayed at the extreme left of the chart. Then the first quartile is at the left of the chart. The median is in the very centre of the chart. The third quartile is at the very right of the chart and then the maximum is the largest number in the set of data and is at the extreme right of the chart.  My AnalysisIn order for me to compete this assignment, there were a number of resources I needed. These included:* Jupyter Notebook* Github* Python version 3.6 downloaded through Anaconda3After some research I realised that I would also need to ensure that there was a number of other libraries installed on Python so that I would be able to complete this assignment. These included matplotlib, numpy and pandas. From that I then had to ensure that other modules and functions were installed which would help me to extract the results from the Met Eireann Temperature Data set and also give me the ability to create different types of charts along with the box plot which I will use to display the data:<jupyter_code>import pandas as pd 
from pandas.plotting import scatter_matrix
import matplotlib.pyplot as plt  
import seaborn as sns  
import numpy as np  


import seaborn as sns
import matplotlib.pyplot as plt

%matplotlib inline

data = [79,44,89,1,13,69,84,100,98,75,91]

r = sns.boxplot(x=data)
r<jupyter_output><empty_output><jupyter_text>Demonstrate the use of a box plot using data of your choiceI am going to use the daily maximum temperature measurements of Athenry weather station from 1st January 2016 to 31st October 2018.This data has been published by Met Éireann and is available to view on https://data.gov.ie/dataset/athenry-daily-weather-station-datatemperature - maxtemp in degrees celsius<jupyter_code>import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
from scipy import stats
import pandas as pd
from pandas import DataFrame

(df) = pd.read_csv (r'C:\Users\riche\Desktop\BoxPlots\athenrytemps.csv')

df['date'] = pd.to_datetime(df['date'])
df

# Set the date column as the index
df.index = df['date'] 

# create a new column for the month
df['month'] = df['date'].dt.strftime('%b')

# create a new column for the year
df['year'] = df['date'].dt.strftime('%Y')

df
df.describe(include='all')
df.groupby("date").describe()
%matplotlib inline

fig, ax = plt.subplots()
fig.set_size_inches((26,30))
plot = sns.boxplot(x = df['year'], y = df['maxtemp'])

plot.set_title('Yearly Temperature Comparison in Athenry 2016-2018',fontsize=20)
plot.set_ylabel('Temperature',fontsize=20)
plot.set_xlabel('Year',fontsize=20)

plt.show()<jupyter_output><empty_output><jupyter_text>Explain any relevant terminology such as the terms quartile and percentile.  PercentileThe median, first quartile and third quartile can all be stated in terms of percentiles. Since half of the data is less than the median, and one-half is equal to 50%, we could call the median the 50th percentile. One-fourth is equal to 25%, and so the first quartile the 25th percentile. Similarly, the third quartile is the same as the 75th percentile. Inter-quartile rangeThe middle box which we can see on the above diagram represents 50% of the data. This range of data from the lower to the upper quartile is referred to as the inter-quartile range. Upper QuartileSeventy-five percent of the data will fall below the upper quartile. This is located between the end of the box and the upper whisker. Lower QuartileTwenty-five percent of the data will also fall below the upper quartile. This is located between the bottom of the box and the lower whisker. WhiskersShows the entire spread of the data. The whiskers display the highest(upper whisker) and lowest points (lower whisker) on the box plot. OutlierOutliers that are located beyond the whiskers tend to represent the extremes. They are treated as abnormal values which tend to represent extremely high or extremely low data values.  Compare the box plot to alternatives. The four plot alternatives which I am going to compare the box plot to are:* Histogram* Violin Plot* Strip Plot* Swarm Plot          Histogram A Histogram is a graphical display of data using bars of different heights. It is similar to a Bar Chart, but a histogram groups numbers into ranges.[1]Histograms appeared to be getting a lot of mentions in numerous articles which I read so I thought this would be the best and main way to display my data in a graphic format. I was unsure of how to go about this as it was not something I had done previously, however I stumbled across an article on matplotlib [2] which went through in detail, how to go about creating Histograms in Python.<jupyter_code>df.hist('maxtemp') #plot my histogram

plt.title('Histogram of the Yearly Temperature Comparison in Athenry 2016-2018') #title of my histogram

plt.xlabel('Temperature in Degrees') #x axis label

plt.ylabel('Number of Days this Temp occured') #y axis label

plt.savefig('project_hist_maxtemp.png') #save the histogram

plt.show() #show my histogram<jupyter_output><empty_output><jupyter_text>Violin Plot The violin plot is usually used to show the distribution of a specific set of data along with its probability density. The violin plot is a mix of a Density Plot and a Box Plot which is rotated and placed on all sides which displays the distribution of the data.[3] We can see below that there is a black bar in the middle which shows us the interquartile range and a skinny line which shows us the 95% confidence intervals and then we can also see a while dot in the below graph – this represents the median. While Violin Plots show us more information than your standard Box Plot they tend to be more in louder than a box plot.<jupyter_code>%matplotlib inline

fig, ax = plt.subplots()
fig.set_size_inches((10,10))
plot = sns.violinplot(x = df['year'], y = df['maxtemp'])

plot.set_title('Yearly Temperature Comparison in Athenry 2016-2018',fontsize=20)
plot.set_ylabel('Temperature',fontsize=20)
plot.set_xlabel('Year',fontsize=20)

plt.show()<jupyter_output>C:\Users\riche\Anaconda4\lib\site-packages\scipy\stats\stats.py:1713: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  return np.add.reduce(sorted[indexer] * weights, axis=axis) / sumval<jupyter_text>Strip Plot When you are working with a data set which is not too big a strip plot can be useful as it is visually appeasing to the eye. It just displays the entire data points from the set. It can be very useful when it comes to displaying visualisations of dispersed data within the outliers. The easiest way to think of a strip plot would be to think of a scatter plot where one specific variable in the data set is defined as categorical. As we can see from the below we had the most extreme temperatures with a number of dates reaching over 30 degrees and a number of dates dropping below 0 degrees.<jupyter_code>%matplotlib inline

fig, ax = plt.subplots()
fig.set_size_inches((10,10))

ax = sns.stripplot(x = df['year'], y = df['maxtemp'],  jitter=True)<jupyter_output><empty_output><jupyter_text>Swarm Plot From the above we can see that in particular, one potential problem with a strip plot is that there is very dense grouping of data points specifically around the 15 degree mark, this can cause data points being plotted over top of one another on the chart and can lead to the data becoming obscure.The Swarm Plot improves upon the random jittering approach to move data points the minimum distance away from one another to avoid “over-plotting”. The result is a plot where there is a distinct data point, this can be seen in the below graph. It can look like a swarm of bees buzzing around a hive. We can also see that each data point is clearly visible and no data is being obscured by being plotted on top of one another. This was in my favourite plot to graph.<jupyter_code>%matplotlib inline

fig, ax = plt.subplots()
fig.set_size_inches((10,10))

sns.swarmplot(x = df['year'], y = df['maxtemp'])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.053244926662648184, 'fraction_numerical': 0.012758689973879847, 'mean_word_length': 4.80466472303207, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 26, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '168049', 'license': 'Apache-2.0', 'n_tokens_mistral': 2769, 'n_tokens_neox': 2572, 'n_words': 1492, 'path': 'Fundamentals of Data Analysis Project 2018.ipynb', 'repo_name': 'richellaod/Fundamentals-of-Data-Analysis-Project-2018', 'size': '9973'}