<jupyter_start><jupyter_text>Introduction to Seaborn About the DataIn this notebook, we will be working with 2 datasets:- Facebook's stock price throughout 2018 (obtained using the [`stock_analysis` package](https://github.com/stefmolin/stock-analysis))- Earthquake data from September 18, 2018 - October 13, 2018 (obtained from the US Geological Survey (USGS) using the [USGS API](https://earthquake.usgs.gov/fdsnws/event/1/)) Setup<jupyter_code>%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns

fb = pd.read_csv(
    'data/fb_stock_prices_2018.csv', 
    index_col='date', 
    parse_dates=True
)
quakes = pd.read_csv('data/earthquakes.csv')<jupyter_output><empty_output><jupyter_text>Categorical dataA [7.5 magnitude earthquake on September 28, 2018 near Palu, Indonesia caused a devastating tsunami afterwards](https://www.livescience.com/63721-tsunami-earthquake-indonesia.html). Let's take a look at some visualizations to understand which magnitude techniques (`magType`) are used in Indonesia, the range of magnitudes there, and how many of the earthquakes are accompanied by a tsunami.<jupyter_code>quakes\
    .assign(
        time=lambda x: pd.to_datetime(x.time, unit='ms')
    )\
    .set_index('time')\
    .loc['2018-09-28']\
    .query(
        'parsed_place == "Indonesia" and tsunami and mag == 7.5'
    )<jupyter_output><empty_output><jupyter_text>`stripplot()`The `stripplot()` function helps us visualize categorical data on one axis and numerical data on the other. We also now have the option of coloring our points using a column of our data (with the `hue` parameter). Using a strip plot, we can see points for each earthquake that was measured with a given `magType` and what its magnitude was; however, it isn't too easy to see density of the points due to overlap:<jupyter_code>sns.stripplot(
    x='magType',
    y='mag',
    hue='tsunami',
    data=quakes.query('parsed_place == "Indonesia"')
)<jupyter_output><empty_output><jupyter_text>`swarmplot()`The bee swarm plot helps address this issue by keeping the points from overlapping. Notice how many more points we can see for the blue section of the `mb` column:<jupyter_code>sns.swarmplot(
    x='magType',
    y='mag',
    hue='tsunami',
    data=quakes.query('parsed_place == "Indonesia"'),
    size=3.5 # point size
)<jupyter_output><empty_output><jupyter_text>`boxenplot()`The `boxenplot()` function creates an enhanced box plot that shows additional quantiles:<jupyter_code>sns.boxenplot(
    x='magType', 
    y='mag', 
    data=quakes[['magType', 'mag']]
)
plt.title('Comparing earthquake magnitude by magType')<jupyter_output><empty_output><jupyter_text>*Read more about this plot type in this paper: [Letter-value plots: Boxplots for large data](https://vita.had.co.nz/papers/letter-value-plot.html)* `violinplot()`Violin plots combine box plots and KDEs:<jupyter_code>fig, axes = plt.subplots(figsize=(10, 5))

sns.violinplot(
    x='magType', 
    y='mag', 
    data=quakes[['magType', 'mag']],  
    ax=axes, 
    scale='width' # all violins have same width
)

plt.title('Comparing earthquake magnitude by magType')<jupyter_output><empty_output><jupyter_text>Correlations and Heatmaps `heatmap()`Last chapter, we saw how to create a correlation matrix heatmap using `pandas` and `matplotlib` (in the [`2-plotting_with_pandas.ipynb`](../ch_05/2-plotting_with_pandas.ipynb) notebook). An easier way is to use `seaborn`:<jupyter_code>sns.heatmap(
    fb\
        .sort_index()\
        .assign(
            log_volume=np.log(fb.volume),
            max_abs_change=fb.high - fb.low
        )\
        .corr(),
    annot=True, 
    center=0, 
    vmin=-1, 
    vmax=1
)<jupyter_output><empty_output><jupyter_text>`pairplot()`The pair plot is seaborn's answer to the scatter matrix we saw in chapter 5 (`pandas.plotting.scatter_matrix()`) in the [`3-pandas_plotting_module.ipynb`](../ch_05/3-pandas_plotting_module.ipynb) notebook:<jupyter_code>sns.pairplot(fb)<jupyter_output><empty_output><jupyter_text>Just as with `pandas` we can specify what to show along the diagonal; however, `seaborn` also allows us to color the data based on another column (or other data with the same shape):<jupyter_code>sns.pairplot(
    fb.assign(quarter=lambda x: x.index.quarter),
    diag_kind='kde',
    hue='quarter'
)<jupyter_output><empty_output><jupyter_text>`jointplot()`The joint plot allows us to visualize the relationship between two variables, like a scatter plot. However, we get the added benefit of being able to visualize their distributions at the same time (as a histogram or KDE). The default options give us a scatter plot in the center and histograms on the sides:<jupyter_code>sns.jointplot(
    x='log_volume',
    y='max_abs_change',
    data=fb.assign(
        log_volume=np.log(fb.volume),
        max_abs_change=fb.high - fb.low
    )
)<jupyter_output><empty_output><jupyter_text>We can pass `kind='hex'` for hexbins:<jupyter_code>sns.jointplot(
    x='log_volume',
    y='max_abs_change',
    kind='hex',
    data=fb.assign(
        log_volume=np.log(fb.volume),
        max_abs_change=fb.high - fb.low
    )
)<jupyter_output><empty_output><jupyter_text>If we pass `kind='kde'`, we get a contour plot of the joint density estimate with KDEs along the sides:<jupyter_code>sns.jointplot(
    x='log_volume',
    y='max_abs_change',
    kind='kde',
    data=fb.assign(
        log_volume=np.log(fb.volume),
        max_abs_change=fb.high - fb.low
    )
)<jupyter_output><empty_output><jupyter_text>If we specify `kind='reg'` instead, we get a regression line in the center and KDEs on the sides:<jupyter_code>sns.jointplot(
    x='log_volume',
    y='max_abs_change',
    kind='reg',
    data=fb.assign(
        log_volume=np.log(fb.volume),
        max_abs_change=fb.high - fb.low
    )
)<jupyter_output><empty_output><jupyter_text>If we pass `kind='resid'`, we get the residuals from the aforementioned regression:<jupyter_code>sns.jointplot(
    x='log_volume',
    y='max_abs_change',
    kind='resid',
    data=fb.assign(
        log_volume=np.log(fb.volume),
        max_abs_change=fb.high - fb.low
    )
)

# update y-axis label (discussed in the next notebook)
plt.ylabel('residuals')<jupyter_output><empty_output><jupyter_text>Regression plotsWe are going to use `seaborn` to visualize a linear regression between the log of the volume traded in Facebook stock and the maximum absolute daily change (daily high stock price - daily low stock price). To do so, we first need to isolate this data:<jupyter_code>fb_reg_data = fb\
    .assign(
        log_volume=np.log(fb.volume),
        max_abs_change=fb.high - fb.low
    )\
    .iloc[:,-2:]<jupyter_output><empty_output><jupyter_text>Since we want to visualize each column as the regressor, we need to look at permutations of their order. Permutations and combinations (among other things) are made easy in Python with `itertools`, so let's import it:<jupyter_code>import itertools<jupyter_output><empty_output><jupyter_text>`itertools` gives us efficient iterators. Iterators are objects that we loop over, exhausting them. This is an iterator from `itertools`; notice how the second loop doesn't do anything:<jupyter_code>iterator = itertools.repeat("I'm an iterator", 2)

for i in iterator:
    print(f'-->{i}')

print('This printed once because the iterator has been exhausted')

for i in iterator:
    print(f'-->{i}')<jupyter_output>-->I'm an iterator
-->I'm an iterator
This printed once because the iterator has been exhausted<jupyter_text>Iterables are objects that can be iterated over. When entering a loop, an iterator is made from the iterable to handle the iteration. Iterators are iterables, but not all iterables are iterators. A list is an iterable. If we turn that iterator into an iterable (a list in this case), the second loop runs:<jupyter_code>iterable = list(itertools.repeat("I'm an iterable", 2))

for i in iterable:
    print(f'-->{i}')

print('This prints again because it\'s an iterable:')
for i in iterable:
    print(f'-->{i}')<jupyter_output>-->I'm an iterable
-->I'm an iterable
This prints again because it's an iterable:
-->I'm an iterable
-->I'm an iterable<jupyter_text>The `reg_resid_plots()` function from the `viz.py` module in this folder uses `regplot()` and `residplot()` from seaborn along with `itertools` to plot the regression and residuals side-by-side:<jupyter_code>from viz import reg_resid_plots
reg_resid_plots??<jupyter_output>[0;31mSignature:[0m [0mreg_resid_plots[0m[0;34m([0m[0mdata[0m[0;34m)[0m[0;34m[0m[0;34m[0m[0m
[0;31mSource:[0m   
[0;32mdef[0m [0mreg_resid_plots[0m[0;34m([0m[0mdata[0m[0;34m)[0m[0;34m:[0m[0;34m[0m
[0;34m[0m    [0;34m"""[0m
[0;34m    Using `seaborn`, plot the regression and residuals plots[0m
[0;34m    side-by-side for every permutation of 2 columns in the data.[0m
[0;34m    [0m
[0;34m    Parameters:[0m
[0;34m        - data: A `pandas.DataFrame`[0m
[0;34m[0m
[0;34m    Returns:[0m
[0;34m        A matplotlib `Axes` object.[0m
[0;34m    """[0m[0;34m[0m
[0;34m[0m    [0mnum_cols[0m [0;34m=[0m [0mdata[0m[0;34m.[0m[0mshape[0m[0;34m[[0m[0;36m1[0m[0;34m][0m[0;34m[0m
[0;34m[0m    [0mpermutation_count[0m [0;34m=[0m [0mnum_cols[0m [0;34m*[0m [0;34m([0m[0mnum_cols[0m [0;34m-[0m [0;36m1[0m[0;34m)[0m[0;34m[0m
[0;34m[0m[0;34m[0m
[0;34m[0m    [0mfig[0m[0;34m,[0m [0max[0m [0;34m=[0m [0mplt[0m[...]<jupyter_text>Let's see what the output looks like for the Facebook data we isolated:<jupyter_code>from viz import reg_resid_plots
reg_resid_plots(fb_reg_data)<jupyter_output><empty_output><jupyter_text>We can use `lmplot()` to split our regression across subsets of our data. For example, we can perform a regression per quarter on the Facebook stock data:<jupyter_code>sns.lmplot(
    x='log_volume',
    y='max_abs_change',
    data=fb.assign(
        log_volume=np.log(fb.volume),
        max_abs_change=fb.high - fb.low,
        quarter=lambda x: x.index.quarter
    ),
    col='quarter'
)<jupyter_output><empty_output><jupyter_text>FacetingWe can create subplots across subsets of our data by faceting. First, we create a `FacetGrid` specifying how to lay out the plots (which categorical column goes along the rows and which one along the columns). Then, we call the `map()` method of the `FacetGrid` and pass in the plotting function we want to use (along with any additional arguments).Let's use a facet grid to show the distribution of earthquake magnitudes in Indonesia and Papua New Guinea by `tsunami` and `parsed_place`:<jupyter_code>g = sns.FacetGrid(
    quakes.query(
        'parsed_place.isin(["Indonesia", "Papua New Guinea"]) '
        'and magType == "mb"'
    ),
    row='tsunami',
    col='parsed_place',
    height=4
)
g = g.map(
    sns.histplot, 
    'mag', 
    kde=True
)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.128212162780064, 'fraction_numerical': 0.02450845907636031, 'mean_word_length': 4.749737118822292, 'pattern_counts': {'":': 0, '<': 86, '<?xml version=': 0, '>': 96, 'https://': 4, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '22', 'dir': 'jupyter-structured-clean-dedup', 'id': '488349', 'license': 'MIT', 'n_tokens_mistral': 4181, 'n_tokens_neox': 3800, 'n_words': 1024, 'path': 'ch_06/1-introduction_to_seaborn.ipynb', 'repo_name': 'will-i-amv-books/Hands-On-Data-Analysis-with-Pandas-2nd-edition', 'size': '10935'}