<jupyter_start><jupyter_text>MMA 831 DOS1This assignment requires R but is good Python practice.  Some more visualizations using Python + Seaborn<jupyter_code>import numpy as np

import pandas as pd

import matplotlib as mpl
import matplotlib.pyplot as plt
plt.set_cmap('Set2')

import seaborn as sns
colours = sns.color_palette('Set2')
plt.rcParams["figure.figsize"] = (15,15)<jupyter_output><empty_output><jupyter_text>Load Data<jupyter_code>dat = pd.read_csv('adv_sales.csv')<jupyter_output><empty_output><jupyter_text>Summary statistics etc.<jupyter_code>dat.shape
dat.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 1000 entries, 0 to 999
Data columns (total 8 columns):
Unnamed: 0    1000 non-null int64
store         1000 non-null int64
billboard     1000 non-null int64
printout      1000 non-null int64
sat           1000 non-null int64
comp          1000 non-null int64
price         1000 non-null int64
sales         1000 non-null float64
dtypes: float64(1), int64(7)
memory usage: 62.6 KB<jupyter_text>Looks like there's an Store ID column, let's delete that, not important.<jupyter_code>dat_noids = dat.drop('Unnamed: 0', axis = 1)
dat_noids.describe()<jupyter_output><empty_output><jupyter_text>Looks like no outliers from here.<jupyter_code>dat_noids.head()<jupyter_output><empty_output><jupyter_text>Check for missing values<jupyter_code>dat_noids.isnull().sum()<jupyter_output><empty_output><jupyter_text>No missing values.  Visualizations Here comes the fancy stuff!  Histograms<jupyter_code>dat_noids.hist(bins = 20, figsize = (20,20))
plt.show()<jupyter_output><empty_output><jupyter_text>Looks to be mostly normal, except for price and satisfaction.  Pair plotsUse `diag_kind = 'kde'` to show kernel density estimates in the diagonals.`kind = reg` shows regression lines for each scatterplot. This increases run time by A LOT. Can also use a correlation matrix (next plot) to visualize pairwise linear relationships.<jupyter_code>sns.pairplot(dat_noids, palette = colours, kind = 'reg', diag_kind = 'kde')
plt.show()<jupyter_output><empty_output><jupyter_text>Correlation matrixIf wanted can draw a white mask to only show half of the plot (because each half is the same), ~~but I'm too lazy~~ I did it anyway but prefer the old one more :grin:First we define a diverging colour palette:<jupyter_code>diverging_palette = sns.diverging_palette(220, 10, as_cmap = True)<jupyter_output><empty_output><jupyter_text>Plot it:Make sure to include `vmin` and `vmax` or you will get a bunch of blue cells that tell you nothing...<jupyter_code># sns.heatmap(dat_noids.corr(), cmap = diverging_palette, square = True, annot = True)
# mask = np.zeros_like(dat_noids.corr(), dtype=np.bool)
# mask[np.triu_indices_from(mask)] = True
sns.heatmap(dat_noids.corr(), cmap = diverging_palette, square = True, annot = True,
           vmin = -1, vmax = 1)
plt.show()<jupyter_output><empty_output><jupyter_text>Aside from `store` and `billboard`, no strong correlation. `price` and `sat` have weak correlation, which is understandable.  Plot scatterplots for advertising channels<jupyter_code>fig, ax = plt.subplots(1, 3, figsize = (15,5), sharey = True)
sns.scatterplot(x = 'store', y = 'sales', data = dat_noids, ax = ax[0])
sns.scatterplot(x = 'billboard', y = 'sales', data = dat_noids, ax = ax[1])
sns.scatterplot(x = 'printout', y = 'sales', data = dat_noids, ax = ax[2])
fig.show()<jupyter_output><empty_output><jupyter_text>Alternatively put it all on the same graph.(This took me an hour to figure out. Thanks Google and StackOverflow!)<jupyter_code>fig, ax = plt.subplots(figsize = (10,10))
ax.grid(True, alpha = 0.5)
ax.scatter(x = 'store', y = 'sales', data = dat_noids, color = '#66c2a5')
ax.scatter(x = 'printout', y = 'sales', data = dat_noids, color = '#fc8d62')
ax.scatter(x = 'billboard', y = 'sales', data = dat_noids, color = '#8da0cb')
ax.set_title('Advertising vs. Sales')
ax.set_ylabel('Sales')
ax.set_xlabel('Advertising')
ax.legend(['Store', 'Printout', 'Billboard'], loc = 4, fontsize = 'large')
fig.show()<jupyter_output><empty_output><jupyter_text>Alternatively, do a pairplot with only the three interesting features:<jupyter_code>sns.pairplot(dat_noids[['sales', 'store', 'billboard', 'printout']], palette = colours)
plt.show()<jupyter_output><empty_output><jupyter_text>Split Train and Test<jupyter_code># TODO<jupyter_output><empty_output><jupyter_text>Linear Regression<jupyter_code># TODO<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12218003127094036, 'fraction_numerical': 0.025910207728389548, 'mean_word_length': 6.1648, 'pattern_counts': {'":': 0, '<': 61, '<?xml version=': 0, '>': 61, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '7865', 'license': 'MIT', 'n_tokens_mistral': 1693, 'n_tokens_neox': 1562, 'n_words': 414, 'path': 'MMA831/DOS1/dos1_python.ipynb', 'repo_name': 'uncrtv/MMA2020', 'size': '4477'}