<jupyter_start><jupyter_text>Heat MapsA heat map is a 2-d version of the histogram that can be used as an alternative to a scatterplot. Like a scatterplot, the values of the two numeric variables to be plotted are placed on the plot axes. Similar to a histogram, the plotting area is divided into a grid and the number of points in each grid rectangle is added up. Since there won't be room for bar heights, counts are indicated instead by grid cell color. A heat map can be implemented with Matplotlib's hist2d function.  Original Plot using Jitter and Transparency<jupyter_code>sb.regplot(data=fuel_econ, x ='displ', y = 'comb', x_jitter=0.04, scatter_kws={'alpha': 1/10}, fit_reg=False);<jupyter_output><empty_output><jupyter_text>Comparison using heatmap<jupyter_code>plt.hist2d(data=fuel_econ, x ='displ', y = 'comb');
plt.colorbar(); #for legend
plt.xlabel("Displacement");
plt.ylabel("Combined Fuel Eff");<jupyter_output><empty_output><jupyter_text>Improvement -- Set minimum count before area can be colored w/ cmin<jupyter_code>plt.hist2d(data=fuel_econ, x ='displ', y = 'comb', cmin=0.5);
plt.colorbar(); #for legend
plt.xlabel("Displacement");
plt.ylabel("Combined Fuel Eff");
# reverse color map since light (white) means fewer
plt.hist2d(data=fuel_econ, x ='displ', y = 'comb', cmin=0.5, cmap='viridis_r');
plt.colorbar(); #for legend
plt.xlabel("Displacement");
plt.ylabel("Combined Fuel Eff");
# Change bin sizes 
bins_x = np.arange(0.6, 7+0.3, 0.3)
bins_y = np.arange(12, 58+3, 3)
plt.hist2d(data=fuel_econ, x ='displ', y = 'comb', cmin=0.5, cmap='viridis_r', bins=[bins_x, bins_y]);
plt.colorbar(); #for legend
plt.xlabel("Displacement");
plt.ylabel("Combined Fuel Eff");<jupyter_output><empty_output><jupyter_text>Adding Counts to HeatmapIf you have a lot of data, you might want to add annotations to cells in the plot indicating the count of points in each cell. From hist2d, this requires the addition of text elements one by one, much like how text annotations were added one by one to the bar plots in the previous lesson. We can get the counts to annotate directly from what is returned by hist2d, which includes not just the plotting object, but an array of counts and two vectors of bin edges.<jupyter_code># hist2d returns a number of different variables, including an array of counts
bins_x = np.arange(0.6, 7+0.3, 0.3)
bins_y = np.arange(12, 58+3, 3)
h2d = plt.hist2d(data=fuel_econ, x ='displ', y = 'comb', cmin=0.5, cmap='viridis_r', bins=[bins_x, bins_y]);
plt.colorbar(); #for legend
plt.xlabel("Displacement");
plt.ylabel("Combined Fuel Eff");
counts = h2d[0]

# loop through the cell counts and add text annotations for each
for i in range(counts.shape[0]):
    for j in range(counts.shape[1]):
        c = counts[i,j]
        if c >= 7: # increase visibility on darkest cells
            plt.text(bins_x[i]+0.5, bins_y[j]+0.5, int(c),
                     ha = 'center', va = 'center', color = 'white')
        elif c > 0:
            plt.text(bins_x[i]+0.5, bins_y[j]+0.5, int(c),
                     ha = 'center', va = 'center', color = 'black')
ct_counts = fuel_econ.groupby(["make", "fuelType"]).size()
ct_counts = ct_counts.reset_index(name="count")
ct_counts.pivot(index="fuelType", columns="make", values='count')
# remove NaN
ct_counts.fillna(value=np.nan, inplace=True)
ct_counts
sb.heatmap(ct_counts);
import seaborn as sns
import numpy as np
import pandas as pd

df = pd.DataFrame(data={'a': [1, 1, 1],
                        'b': [2, np.nan, 2],
                        'c': [3, 3, np.nan]})

sns.heatmap(df)
import seaborn as sns
import numpy as np
import pandas as pd

df = pd.DataFrame(data={'a': [1, 1, 1],
                        'b': [2, np.nan, 2],
                        'c': [3, 3, np.nan]})

mask = df.isnull()
sns.heatmap(df, mask=mask)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13319404544267432, 'fraction_numerical': 0.02063201880386524, 'mean_word_length': 4.1897018970189706, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 19, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '242826', 'license': 'MIT', 'n_tokens_mistral': 1427, 'n_tokens_neox': 1320, 'n_words': 436, 'path': 'scratch/lesson_3/matplotlib_seaborn/heatmap.ipynb', 'repo_name': 'jeremybeasley/udacity_aipnd', 'size': '3829'}