<jupyter_start><jupyter_text>Data Visualization in Python* There are many libraries for doing data visualization in Python, none of them are perfect* matplotlib is a very powerful data visualization toolkit for Python* But it is a bit clunky and old school* Increasingly there are newer alternatives to graphing in Python    * [Seaborn](http://seaborn.pydata.org/), built on top of matplotlib, provides an easier (and more asthetically pleasing) programming interface     * [Bokeh](http://bokeh.pydata.org/en/latest/), which is great for displaying data on the web    * [Plotly](https://plot.ly/python/), which is a whole subscription based web service    * [plotnine](https://plotnine.readthedocs.io/en/stable/), a python implementation of the very popular visualization library GGPlot from R    * [Altair](https://altair-viz.github.io), which is a new *declarative* visualization library from one of the Jupyter developers* Still, `matplotlib` is the OG of the python data visualization libraries (respect), so we are going to spend the next couple hours getting matplotlib'd  Visualizing data* I assume you already know what data visualization *is** The *real* question is *how* to visualize data AND *what* visualization to use* There are many different kinds of data visualizations and which one is the best for any given situation is a loaded question![XKCD comic on data vis](https://imgs.xkcd.com/comics/self_description.png) What kinds of visulizations are there?* Here is a semy ugly looking periodic table of visualization methods: [http://www.visual-literacy.org/periodic_table/periodic_table.html](http://www.visual-literacy.org/periodic_table/periodic_table.html)* Here is a good website cataloguing many of the different types of visualizations: [http://www.datavizcatalogue.com](http://www.datavizcatalogue.com)* For example, here is the information about [box charts](http://www.datavizcatalogue.com/methods/bar_chart.html)* When making a data visualization, you need to consider how various visual elements can convey the information you wish to show.* There are many ways to represent quantiative information visually  Visualizing Data<jupyter_code># load up our python libraries
# all will be explained later
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
plt.style.use('ggplot')

# create a bunch of random data
data = np.random.randn(1000)
data[0:10]<jupyter_output><empty_output><jupyter_text>Univariate or one dimensional data* Histograms are useful because they show you how the values are distributed* Not to be confused with bar charts!<jupyter_code># create a histogram of the data
plt.hist(data);<jupyter_output><empty_output><jupyter_text>Bivariate or two dimensional data * Scatter plots are useful for visualizing two dimensional numerical data<jupyter_code># create two lists of numerical data
x = np.linspace(0, 10, 30)
y = np.sin(x)

# plot the data as a series of points
plt.plot(x, y, 'o');<jupyter_output><empty_output><jupyter_text>* Bar charts are useful for visualizing two dimensional data where one dimension is categorical<jupyter_code># create some categorical data
labels = ["cats", "dogs", "chickens", "spiders"]
x_pos = [1,2,3,4]
y = [1,5,2,5]

# plot as a bar chart
plt.bar(x_pos, y)
plt.xticks(x_pos, labels);<jupyter_output><empty_output><jupyter_text>* And line charts can be used to represent continuous data<jupyter_code>plt.plot(x, np.sin(x));<jupyter_output><empty_output><jupyter_text>* So this plot has two dimensions* How might we add more?  Adding more dimensions?* What if we want to visualize additional dimensions? * Color, size, and shapes add information from other dimensions<jupyter_code>rng = np.random.RandomState(0)
x = rng.randn(100)
y = rng.randn(100)
colors = rng.rand(100)
sizes = 1000 * rng.rand(100)

plt.scatter(x, y, c=colors, s=sizes, alpha=0.3,
            cmap='viridis')
plt.colorbar();  # show color scale<jupyter_output><empty_output><jupyter_text>* How many dimensions are we displaying in this visualization?  Interacting with matplotlib* Two interfaces (or "data structures" to be consistent with how we've been talking in these workshops)    * `pyplot` - which is designed to resemble the *not pythonic* plotting interface of MATLAB        * You programmatically issue commands to build your plot    * The "object oriented" and more *pythonic* interfaces, consisting of         * `Figure` - a "data structure" that contains the figure. Has parameters and methods for setting the physical size of the visualzations, saving the visualization as an image to disk, etc. Usually stored in the `fig` variable.        * `Axes` - a "data structure" that contains the specific elements of the visualization. You can have more than one axes if you want to create parallel plots. Has parameters and methods for setting the tick marks, legend, axis labels, etc. Usually stored in the `ax` variable.  matplotlib and Jupyter Notebooks* In order to use `matplotlib` with Jupyter Notebooks, you need to run this "magic" command    * It is best to put it at the top of your notebook when you import libraries* This command tells matplotlib to render the plots inside the notebook<jupyter_code>%matplotlib inline<jupyter_output><empty_output><jupyter_text>* The convention is to import the pyplot interface as `plt`<jupyter_code>import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>* Now whenever you want to create a data vizualization, you will methods part of the `plt` "data structure"<jupyter_code># create two lists of numerical data
x = np.linspace(0, 10, 30)
y = np.sin(x)

# plot the data as a series of points
# the 'o' is a format string saying render as points
plt.plot(x, y, 'o');<jupyter_output><empty_output><jupyter_text>* Without that `'o'` format string pyplot defaults to lines* For more examples of format strings see the [matplotlib documentation](https://matplotlib.org/api/_as_gen/matplotlib.pyplot.plot.html)<jupyter_code># create two lists of numerical data
x = np.linspace(0, 10, 30)
y = np.sin(x)

# plot the data as a series of points
plt.plot(x, y);<jupyter_output><empty_output><jupyter_text>* You can make subsequent calls using the `plt` object to modify your chart* Note: Juptyter makes a new plot with each cell, so you must put all your commands for an individual chart in the same cell<jupyter_code># create some categorical data
labels = ["cats", "dogs", "chickens", "spiders"]
x_pos = [1,2,3,4]
y = [1,5,2,5]

# plot as a bar chart
plt.bar(x_pos, y);
#plt.xticks(x_pos, labels);
plt.xticks(x_pos, labels);
# create some categorical data
labels = ["cats", "dogs", "chickens", "spiders"]
x_pos = [1,2,3,4]
y = [1,5,2,5]

# plot as a bar chart
plt.bar(x_pos, y);
plt.xticks(x_pos, labels);<jupyter_output><empty_output><jupyter_text>* Even if you use the other interface to build your viz, you start with `plt`  Modifying your Chat<jupyter_code>data = np.random.randn(30).cumsum()
plt.plot(data, 'ko--');<jupyter_output><empty_output><jupyter_text>* Another way of saying this not using a format string<jupyter_code>plt.plot(data, color='k', linestyle='dashed', marker='o');<jupyter_output><empty_output><jupyter_text>* You can add a legend to your chart<jupyter_code>plt.plot(data, color='k', linestyle='dashed', marker='o');
plt.legend(["Plot"], loc='best');<jupyter_output><empty_output><jupyter_text>* You can also set the title, axis labels, ticks, and ticklables using subsequent calls to the `plt` object<jupyter_code>plt.plot(data, color='k', linestyle='dashed', marker='o');
plt.xticks([0,10,20,30]);<jupyter_output><empty_output><jupyter_text>* You can also specify your own labels for the axes<jupyter_code>plt.plot(data, color='k', linestyle='dashed', marker='o');
plt.xticks([0,10,20,30], ["start", "early", "middle", "end"]);<jupyter_output><empty_output><jupyter_text>* There is an equivalent `plt.yticks()` for modifying the y axis.<jupyter_code>plt.plot(data, color='k', linestyle='dashed', marker='o');
plt.xticks([0,10,20,30], ["start", "early", "middle", "end"]);

plt.yticks([-5,-3,0,3,5]);<jupyter_output><empty_output><jupyter_text>* This plot still feels small, so let's make it bigger!<jupyter_code># note, you need to specify this before you run the plot function
plt.figure(figsize=(10,8))
plt.plot(data, color='k', linestyle='dashed', marker='o');
plt.xticks([0,10,20,30], ["start", "early", "middle", "end"]);<jupyter_output><empty_output><jupyter_text>Saving plots to a file<jupyter_code>plt.plot(data, color='k', linestyle='dashed', marker='o');
plt.xticks([0,10,20,30], ["start", "early", "middle", "end"]);

plt.savefig('my-sweet-chart.png')<jupyter_output><empty_output><jupyter_text>* Sweet! But what if we want to make it bigger<jupyter_code>plt.plot(data, color='k', linestyle='dashed', marker='o');
plt.xticks([0,10,20,30], ["start", "early", "middle", "end"]);

plt.savefig('my-sweet-chart-big.png', dpi=400, bbox_inches='tight')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11622708985248101, 'fraction_numerical': 0.013410818059901655, 'mean_word_length': 5.99687255668491, 'pattern_counts': {'":': 0, '<': 81, '<?xml version=': 0, '>': 81, 'https://': 5, 'lorem ipsum': 0, 'www.': 5, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '20', 'dir': 'jupyter-structured-clean-dedup', 'id': '313221', 'license': 'MIT', 'n_tokens_mistral': 2986, 'n_tokens_neox': 2812, 'n_words': 995, 'path': 'day-two/extra-pandas/4 - Data Visualization in Python.ipynb', 'repo_name': 'mcburton/lc-python-training', 'size': '8948'}