<jupyter_start><jupyter_text>_Lambda School Data Science_ Choose appropriate visualizationsObjectives- fix misleading visualizations- use Seaborn to visualize distributions and relationships with continuous and discrete variablesLinks- [How to Spot Visualization Lies](https://flowingdata.com/2017/02/09/how-to-spot-visualization-lies/)- [Visual Vocabulary - Vega Edition](http://ft.com/vocabulary)- [Choosing a Python Visualization Tool flowchart](http://pbpython.com/python-vis-flowchart.html)- [Searborn example gallery](http://seaborn.pydata.org/examples/index.html) & [tutorial](http://seaborn.pydata.org/tutorial.html)  Upgrade SeabornMake sure you have at least version 0.9.0.On Colab, go to **Restart runtime** after you run the `pip` command.<jupyter_code>!pip install --upgrade seaborn
import seaborn as sns
sns.__version__<jupyter_output><empty_output><jupyter_text>Fix misleading visualizations<jupyter_code>%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
!wget https://raw.githubusercontent.com/LambdaSchool/DS-Unit-1-Sprint-3-Data-Storytelling/master/module2-choose-appropriate-visualizations/misleading.py
    
import misleading<jupyter_output>--2019-04-09 20:23:23--  https://raw.githubusercontent.com/LambdaSchool/DS-Unit-1-Sprint-3-Data-Storytelling/master/module2-choose-appropriate-visualizations/misleading.py
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 192.168.127.12, 172.16.58.3, 172.16.58.3, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|192.168.127.12|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1258 (1.2K) [text/plain]
Saving to: ‘misleading.py.1’


misleading.py.1       0%[                    ]       0  --.-KB/s               
misleading.py.1     100%[===================>]   1.23K  --.-KB/s    in 0s      

2019-04-09 20:23:23 (156 MB/s) - ‘misleading.py.1’ saved [1258/1258]<jupyter_text>Fix misleading plot 1<jupyter_code>misleading.plot1();<jupyter_output><empty_output><jupyter_text>The bar for State Farm ought to appear 1.4 times as tall as the bar for GEICO<jupyter_code>18.07 / 12.79<jupyter_output><empty_output><jupyter_text>But it appears almost 3 times as tall, in the misleading plot above<jupyter_code>(18.07 - 10) / (12.79 - 10)
# fix it part 1 - fix the y axis
insurance = pd.DataFrame(
    index=['State Farm', 'GEICO'], 
    data={'Market Share %': [18.07, 12.79]})

insurance.plot.bar()
# fix it part 2- wheres everyone else?
other = 100 - 18.07 - 12.79
other = pd.DataFrame(
    index=['Other'], 
    data={'Market Share %': [100 - 18.07 - 12.79]})

insurance = insurance.append(other)

insurance
insurance.plot.bar()<jupyter_output><empty_output><jupyter_text>Fix misleading plot 2<jupyter_code>misleading.plot2();
# same as before ! We already have our data so lets see what we get
insurance.plot.pie(y='Market Share %', autopct = '%.2f')<jupyter_output><empty_output><jupyter_text>Fix misleading plot 3 _If you're on Jupyter (not Colab) then uncomment and run this cell below:_<jupyter_code># import altair as alt
# alt.renderers.enable('notebook')
misleading.plot3()
# lets make our dataframe of information

url = 'https://raw.githubusercontent.com/LambdaSchool/DS-Sprint-02-Storytelling-With-Data/master/module2-choose-appropriate-visualizations/direct_written_premium_by_state_by_group_private_passenger_auto.csv'
df = pd.read_csv(url)
df.rename(columns={'STATE FARM GRP': 'State Farm policy premiums'}, inplace=True)
df.head()
# lets make a column of normalized data!
df['State Farm market share %'] = df['State Farm policy premiums'] / df['STATE TOTAL']
df.head()
import altair as alt
from vega_datasets import data
states = alt.topo_feature(data.us_10m.url, 'states')
pop_eng_hur = data.population_engineers_hurricanes.url
variable = 'population'

alt.Chart(states).mark_geoshape().encode(
    alt.Color(variable, type='quantitative')
).transform_lookup(
    lookup='id',
    from_=alt.LookupData(pop_eng_hur, 'id', [variable])
).properties(
    width=500,
    height=300
).project(
    type='albersUsa'
)
variable = 'State Farm market share %'

alt.Chart(states).mark_geoshape().encode(
    alt.Color(variable, type='quantitative')
).transform_lookup(
    lookup='id',
    from_=alt.LookupData(df, 'id', [variable])
).properties(
    width=500,
    height=300
).project(
    type='albersUsa'
)<jupyter_output><empty_output><jupyter_text>Use Seaborn to visualize distributions and relationships with continuous and discrete variables  1. Anscombe dataset  Load dataset<jupyter_code>df = sns.load_dataset('anscombe')
df<jupyter_output><empty_output><jupyter_text>See the data's shape<jupyter_code>df.shape<jupyter_output><empty_output><jupyter_text>See the data<jupyter_code>df.describe()
df.head()<jupyter_output><empty_output><jupyter_text>[Group by](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.groupby.html) `'dataset'`<jupyter_code>df_gb = df.groupby('dataset')
df_gb<jupyter_output><empty_output><jupyter_text>[Describe](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.describe.html) the groups<jupyter_code><jupyter_output><empty_output><jupyter_text>Get the [count](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.count.html), for each column in each group<jupyter_code><jupyter_output><empty_output><jupyter_text>Get the [mean](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.mean.html) ...<jupyter_code><jupyter_output><empty_output><jupyter_text>Get the [standard deviation](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.std.html) ...<jupyter_code><jupyter_output><empty_output><jupyter_text>Get the [correlation](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.corr.html) ...<jupyter_code>df.corr()
df_gb.corr()<jupyter_output><empty_output><jupyter_text>Use pandas to [plot](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.plot.html) the groups, as scatter plots<jupyter_code>df_gb.plot.scatter(x='x', y='y')<jupyter_output>'c' argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with 'x' & 'y'.  Please use a 2-D array with a single row if you really want to specify the same RGB or RGBA value for all points.
'c' argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with 'x' & 'y'.  Please use a 2-D array with a single row if you really want to specify the same RGB or RGBA value for all points.
'c' argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with 'x' & 'y'.  Please use a 2-D array with a single row if you really want to specify the same RGB or RGBA value for all points.
'c' argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches[...]<jupyter_text>Use Seaborn to make [relational plots](http://seaborn.pydata.org/generated/seaborn.relplot.html)<jupyter_code>sns.relplot(x='x', y='y', hue='dataset', col = 'dataset', data = df)<jupyter_output><empty_output><jupyter_text>Use Seaborn to make [linear model plots](http://seaborn.pydata.org/generated/seaborn.lmplot.html)<jupyter_code>sns.lmplot(x='x', y='y', col = 'dataset', data=df)<jupyter_output><empty_output><jupyter_text>Links- [Seaborn examples: Anscombe's quartet](http://seaborn.pydata.org/examples/anscombes_quartet.html)- [Wikipedia: Anscombe's quartet](https://en.wikipedia.org/wiki/Anscombe%27s_quartet)- [The Datasaurus Dozen](https://www.autodeskresearch.com/publications/samestats)  2. Tips dataset  Load dataset<jupyter_code>tips = sns.load_dataset('tips')
tips.head(3)<jupyter_output><empty_output><jupyter_text>See the data's shape<jupyter_code>tips.shape<jupyter_output><empty_output><jupyter_text>See the first 5 rows<jupyter_code>tips.head()<jupyter_output><empty_output><jupyter_text>Describe the data<jupyter_code>tips.describe()<jupyter_output><empty_output><jupyter_text>Make univariate [distribution plots](https://seaborn.pydata.org/generated/seaborn.distplot.html)<jupyter_code>sns.distplot(a=tips['total_bill'], bins=10)
tips.sex.replace('Male', 0, inplace=True)
tips.sex.replace('Female', 1, inplace=True)

sns.distplot(tips['sex'])<jupyter_output><empty_output><jupyter_text>Make bivariate [relational plots](https://seaborn.pydata.org/generated/seaborn.relplot.html)<jupyter_code>sns.relplot(x='total_bill', y='tip', hue='sex', data=tips)<jupyter_output><empty_output><jupyter_text>Make univariate [categorical plots](https://seaborn.pydata.org/generated/seaborn.catplot.html)<jupyter_code>sns.catplot(x='size', data=tips, hue = 'sex')
tips.head(4)<jupyter_output><empty_output><jupyter_text>Make bivariate [categorical plots](https://seaborn.pydata.org/generated/seaborn.catplot.html)<jupyter_code>tips['bins'] = pd.cut(tips['total_bill'], 10)
tips
sns.lmplot(x= 'total_bill',
            y='tip',
            data=tips,
            col='time',
            hue = 'sex')<jupyter_output><empty_output><jupyter_text>3. Flights  Load dataset<jupyter_code>flights = sns.load_dataset('flights')<jupyter_output><empty_output><jupyter_text>See the data's shape<jupyter_code>flights.shape<jupyter_output><empty_output><jupyter_text>See the first 5 rows<jupyter_code>flights.head()<jupyter_output><empty_output><jupyter_text>Describe the data<jupyter_code>flights.describe()<jupyter_output><empty_output><jupyter_text>Plot year & passengers<jupyter_code>flights.plot.scatter(x= 'year', y='passengers')<jupyter_output>'c' argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with 'x' & 'y'.  Please use a 2-D array with a single row if you really want to specify the same RGB or RGBA value for all points.<jupyter_text>Plot month & passengers<jupyter_code>import matplotlib.pyplot as plt
flights_gb2 = flights.groupby('month')['passengers'].count()
flights_gb2
flights_gb2.plot(kind='bar')


flights['month']
flights
plt.hist(flights['month']).count<jupyter_output><empty_output><jupyter_text>Create a [pivot table](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.pivot_table.html) of passengers by month and year<jupyter_code>flights = flights.pivot_table(index='year', columns='month', values='passengers')<jupyter_output><empty_output><jupyter_text>Plot the pivot table as a [heat map](https://seaborn.pydata.org/generated/seaborn.heatmap.html)<jupyter_code>sns.heatmap(flights);<jupyter_output><empty_output><jupyter_text>ASSIGNMENT 1.Replicate the lesson code. I recommend that you [do not copy-paste](https://docs.google.com/document/d/1ubOw9B3Hfip27hF2ZFnW3a3z9xAgrUDRReOEo-FHCVs/edit). 2.Choose a chart you want to make, from [Visual Vocabulary - Vega Edition](http://ft.com/vocabulary).Find the chart in an example gallery of a Python data visualization library:- [Seaborn](http://seaborn.pydata.org/examples/index.html)- [Altair](https://altair-viz.github.io/gallery/index.html)- [Matplotlib](https://matplotlib.org/gallery.html)- [Pandas](https://pandas.pydata.org/pandas-docs/stable/visualization.html)Reproduce the chart. [Optionally, try the "<NAME>."](https://docs.google.com/document/d/1ubOw9B3Hfip27hF2ZFnW3a3z9xAgrUDRReOEo-FHCVs/edit) If you want, experiment and make changes.Take notes. Consider sharing your work with your cohort!<jupyter_code>import pandas as pd
df2 = pd.read_csv('heart.csv')
df2
source = df2

bar = alt.Chart(source).mark_bar().encode(
    x='age:O',
    y='chol:Q'
)

rule = alt.Chart(source).mark_rule(color='red').encode(
    y='mean(chol):Q'
)

(bar + rule).properties(width=600)
df2_pd = pd.crosstab(df2['Nationality'], df2['Composure'])
df2_pd.shape
df2['Overall'].max()

data.wheat()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1351868808790477, 'fraction_numerical': 0.020894031465911928, 'mean_word_length': 6.1215174866627144, 'pattern_counts': {'":': 0, '<': 143, '<?xml version=': 0, '>': 144, 'https://': 24, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 4, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '36', 'dir': 'jupyter-structured-clean-dedup', 'id': '291145', 'license': 'MIT', 'n_tokens_mistral': 4518, 'n_tokens_neox': 4191, 'n_words': 839, 'path': 'Copy_of_Copy_of_LS_DS_222_Choose_appropriate_visualizations.ipynb', 'repo_name': 'Jaavion/DS-Unit-1-Sprint-3-Data-Storytelling', 'size': '12031'}