<jupyter_start><jupyter_text>Introduction to Data Viz with Python (Part I)by [<NAME>](https://github.com/eflegara) [![Open In Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/eflegara/Data-Viz-Python/blob/main/Intro%20to%20Data%20Viz%20with%20Python%20%28Part%20I%29.ipynb)Note: _If clicking the *open in colab* badge doesn't work, right-click on the badge and choose to open in new tab._--- In this two-session meeting, we learn about Python's plotting library **[matplotlib](https://matplotlib.org/)**. From its website, it is described as "_a comprehensive library for creating static, animated, and interactive visualizations in Python._"  Here, we cover some of the most fundamental aspects of a matplotlib figure, which—in my opinion—is necessary to gain a better understanding of how we can further customize the look of our visualizations.We will then learn about **seaborn** and **plotly**. They'll be introduced in Part II.Note: If you're super new to Python and/or **matplotlib**, you may want to have someone knowledgeable to handhold you as you go through this notebook, especially since this is meant to be accompanied by a lecture. ---**Let's Begin**First, let's import the libraries. We are already importing `seaborn as sns` as early as now because we will be using some of the datasets in its online repository.<jupyter_code>import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns

%matplotlib inline<jupyter_output><empty_output><jupyter_text>For those new to Python and Jupyter, `%matplotlib` is what we call a magic function in IPython. *Magics* "_are useful as convenient functions where Python syntax is not the most natural one, or when one want to embed invalid python syntax in their work flow_". `%matplotlib inline` "_sets the backend of matplotlib to the 'inline' backend_"; in other words, it allows us, well, to show matplotlib figures inline within the notebooks.   Load DataBelow, let's load our data from the `seaborn` repository. The first dataset we'll use is [Anscombe's quartet](https://en.wikipedia.org/wiki/Anscombe%27s_quartet). It's a classic example many of us use to show the importance of visualizing your data even at the exploratory level, and that the summary statistics, especially those presented as single values, are insufficient to make conclusions about certain systems.<jupyter_code>df_ans = sns.load_dataset('anscombe')
df_ans.head()
df_ans.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 44 entries, 0 to 43
Data columns (total 3 columns):
 #   Column   Non-Null Count  Dtype  
---  ------   --------------  -----  
 0   dataset  44 non-null     object 
 1   x        44 non-null     float64
 2   y        44 non-null     float64
dtypes: float64(2), object(1)
memory usage: 1.2+ KB<jupyter_text>Plotting with matplotlibThere are at least two ways to build vizzes using **matplotlib** based on its two APIs:1. `pyplot` interface2. Object-oriented interface (OOI)The object-oriented API is used for more complex plotting as we will show later; and, the other is through the `pyplot` interface, which is a scripting layer that's state-based. "State-based" means that it takes note of the current `Figure` and the `Axes` objects defined. That is, all changes you implement on a figure will only be implemented a figure (`Axes`) at a time. This can be a bit confusing, but it will get clearer as we move forward with concrete examples. **[pyplot](https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.htmlmodule-matplotlib.pyplot)**, among its other purposes, is intended to "feel" like MATLAB and is mainly used for much simpler visualizations (i.e., less customizability). You may find **pyplot** easier to use compared to **matplotlib**'s OOI, but the latter, as mentioned, is much more powerful. This latter creates and utilizes an `Axes` object. Again, this idea will be much clearer once we dissect a matplotlib figure.But before we deep dive, here's a quick way to plot with `pyplot` as `plt`.<jupyter_code>## The '.' in the argument indicates the marker style
plt.plot(df_ans.x, df_ans.y, '.');<jupyter_output><empty_output><jupyter_text>Basic plot elementsBefore we dive into the **matplotlib objects**, let's have a look first at some of the basic plot elements:- `xlabel` - add x-axis label- `ylabel` - add y-axis label- `xlim` - set x-limits of the x-axis- `ylim` - set y-limits of the y-axis- `legend` - add legend Another thing that we need to know is that the default style of a plot is actually a line. That is, if you draw a datapoint, it will not appear (_since there is no line_) unless you explicitly set the marker style to something visible using `'.'` as illustrated below or other marker styles like `'^'` or `'o'`. Check more markers [here](https://matplotlib.org/stable/api/markers_api.html).<jupyter_code>plt.plot(5, 6);
## Adding the marker '.'
plt.plot(5, 6, '.');<jupyter_output><empty_output><jupyter_text>Now, let's make use of the Anscombe's Quartet dataset.<jupyter_code>## 's' stands for square marker
plt.plot(df_ans.x, df_ans.y, 's');
plt.xlabel('x axis')
plt.ylabel('y axis')
plt.xlim(-2,18)
plt.ylim(-2,15);<jupyter_output><empty_output><jupyter_text>We can also change the colors using special **matplotlib** color abbreviations. **matplotlib** also has a list of other [named colors](https://matplotlib.org/stable/gallery/color/named_colors.html) like "rebeccapurple". Of course, you may also use [hex codes](https://htmlcolorcodes.com/).Below, we show a trick that `pyplot` allows us to do, which is to define color abbreviations with a marker, e.g., `'ro'` stands for red and circle marker, `'bs'` will give us blue square markers.<jupyter_code>## Here we're using circle for markers
## We're also coloring the datapoints red
plt.plot(df_ans.x, df_ans.y, 'ro');
plt.xlabel('x axis')
plt.ylabel('y axis')
plt.xlim(-2,18)
plt.ylim(-2,15);
## Here we're using sqaure for markers
## We're also coloring the datapoints blue
plt.plot(df_ans.x, df_ans.y, 'bs');
plt.xlabel('x axis')
plt.ylabel('y axis')
plt.xlim(-2,18)
plt.ylim(-2,15);<jupyter_output><empty_output><jupyter_text>However, if we want to use other colors (other than the abbreviations), we need to define separately from the marker syle.Notice as well the `linestyle` argument that we have defined as empty. See what happens if you remove that argument.<jupyter_code>## Here we're using triangle for markers
## We're also coloring the datapoints rebecca purple
plt.plot(df_ans.x, df_ans.y, color='rebeccapurple', 
         marker='^', linestyle='');
plt.xlabel('x axis')
plt.ylabel('y axis')
plt.xlim(-2,18)
plt.ylim(-2,15);<jupyter_output><empty_output><jupyter_text>What are the other linestyles?<jupyter_code>## Let's generate a random dataset we call "y"
y = np.random.randn(365).cumsum()
plt.plot(y, color='rebeccapurple', linestyle='--');
y = np.random.randn(365).cumsum()
plt.plot(y, color='crimson', linestyle='-.');
y = np.random.randn(365).cumsum()
plt.plot(y, color='#0F6C09', linestyle=':', label='random');
plt.legend();<jupyter_output><empty_output><jupyter_text>The Object-Oriented interface So far, we have been using the `pyplot` interface which has allowed us to change colors, marker styles, and line styles; we also were able to add axis labels and set axes limits. But what if, for example, we want to perform more modifications on our plots? Say, we want a bigger canvas? Or, what if we want to remove the borders of the plot, and more? Or, if we want subplots and grids?To do this, we need to have to use **matplotlib**'s more powerful object-oriented interface. Here, we formally introduce the `Figure` and `Axes` objects.  The matplotlib Figure( ) instanceIn the image below, we look at the different parts of a `matplotlib` figure [[1](https://matplotlib.org/1.5.3/faq/usage_faq.html)]. Here, we have the `Figure` instance, a figure "child" called the `Axes` and the different components of `Axes` that can also be called as `Axes`'s children. A figure can contain many children `Axes`, but a given `Axes` object can only be in one `Figure`. `Axes` are members of the *artists* group, which also include `Figure`, `title`, `x-label`, `y-label`, the `lines`, etc.Below is another diagram highlighting the different parts of a figure [[1](https://matplotlib.org/faq/usage_faq.html)]. It would be easier to think that a **matplotlib** figure's composed of **encodings**, and that encoding is editable.   Plotting with OOI<jupyter_code>## Still using pyplot
plt.plot(df_ans.x,df_ans.y, '^');
## Here, we explicitly use both Figure and Axes objects
fig, ax = plt.subplots(1,1,figsize=(8,5))
ax.plot(df_ans.x,df_ans.y, '^');<jupyter_output><empty_output><jupyter_text>The `plt.subplots( )` allows us to create a figure (`fig`) and a set of subplots (stored in `ax`). For the specific example above, the "set of subplots" contains only one plot or, more technically, one `Axes`.  Axes ChildrenLet's check the components of our `Axes` object.<jupyter_code>ax.get_children()<jupyter_output><empty_output><jupyter_text>Let's specify the axes limits of the `Axes` object.<jupyter_code>fig, ax = plt.subplots(1,1,figsize=(5,5))
ax.plot(df_ans.x,df_ans.y, '^');
ax.axis([0,20,0,20]);
fig, ax = plt.subplots(1)
ax.plot(18, 9, '.');
ax.plot(11, 8, 'go')
ax.plot(3, 8, 'bo')
ax.axis([0,20,0,10]);
ax.get_children()<jupyter_output><empty_output><jupyter_text>So far, we haven't really shown yet how the OOI approach is more powerful than the `pyplot` interface. But below, we show what else we can do using OOI. In this example, we initially use the `pyplot` interface. And then, we ge the current `Axes` with the `plt.gca()` command. We then store the results in `ax`, which, for this particular example, contains one `Axes`. Using `ax`, we can make some more modifications like removing the top and right spines of our figure (or `Axes` object).<jupyter_code>plt.plot(18, 9, '.');
plt.plot(11, 8, 'go');
plt.plot(3, 8, 'bo');
ax = plt.gca();

ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
ax.spines['bottom'].set_visible(True)
ax.spines['left'].set_visible(True);<jupyter_output><empty_output><jupyter_text>Below, we are generating this same plot, but we're using OOI at the onset by using `fig, ax = plt.subplots(1)`.<jupyter_code>fig, ax = plt.subplots(1)
ax.plot(18, 9, '.');
ax.plot(11, 8, 'go')
ax.plot(3, 8, 'bo')
ax.axis([0,20,0,10]);

ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
ax.spines['bottom'].set_visible(True)
ax.spines['left'].set_visible(True);<jupyter_output><empty_output><jupyter_text>Scatter PlotsIt was mentioned earlier that `plot` actually refers to a line plot. If we really want scatter plots, **matplotlib** has a function that generates those.<jupyter_code>df_ans.head()
fig, ax = plt.subplots(1);

# Plotting x and y for dataset group I
ax.scatter(df_ans[df_ans.dataset=='I']['x'], 
           df_ans[df_ans.dataset=='I']['y'], 
           c = 'r', marker='^');

ax.set_xlabel('x label');
ax.set_ylabel('y label');<jupyter_output><empty_output><jupyter_text>Setting colors<jupyter_code>kulay = ['red']* 9 + ['blue'] * 2
kulay
df_ans[df_ans.dataset=='I']
fig, ax = plt.subplots(1);
x = df_ans[df_ans.dataset=='I']['x']
y = df_ans[df_ans.dataset=='I']['y']

ax.scatter(x, y, c = kulay);
ax.set_xlabel('x label');
ax.set_ylabel('y label');<jupyter_output><empty_output><jupyter_text>Okay, that's a very crude way dealing with colors on matplotlib. We'll cover more on colors later in this notebook.  Grouping and Adding Legends<jupyter_code>fig, ax = plt.subplots(1)

x1 = df_ans[df_ans.dataset=='I']['x']
y1 = df_ans[df_ans.dataset=='I']['y']
x2 = df_ans[df_ans.dataset=='II']['x']
y2 = df_ans[df_ans.dataset=='II']['y']

ax.scatter(x1, y1, c = 'r', label = 'Group I');
ax.scatter(x2, y2, c = 'b', label = 'Group II');

ax.set_xlabel('x label');
ax.set_ylabel('y label');

ax.legend(loc=1);<jupyter_output><empty_output><jupyter_text>What if you want the legend placed outside of the plot as shown below, you can just use the `bbox_to_anchor` argument.<jupyter_code>fig, ax = plt.subplots(1)

x1 = df_ans[df_ans.dataset=='I']['x']
y1 = df_ans[df_ans.dataset=='I']['y']
x2 = df_ans[df_ans.dataset=='II']['x']
y2 = df_ans[df_ans.dataset=='II']['y']

ax.scatter(x1, y1, c = 'r', marker = '^', label = 'Group I');
ax.scatter(x2, y2, c = 'b', marker = 'v', label = 'Group II');

ax.set_xlabel('x label');
ax.set_ylabel('y label');

ax.legend(bbox_to_anchor=(1.3, .9), frameon=False);<jupyter_output><empty_output><jupyter_text>AnnotatingOf course, we can also annotate our figures.<jupyter_code>fig, ax = plt.subplots(1);

x1 = df_ans[df_ans.dataset=='I']['x']
y1 = df_ans[df_ans.dataset=='I']['y']
x2 = df_ans[df_ans.dataset=='II']['x']
y2 = df_ans[df_ans.dataset=='II']['y']

ax.scatter(x1, y1, c = 'r', label = 'Group I');
ax.scatter(x2, y2, c = 'b', label = 'Group II');

ax.set_xlabel('x label');
ax.set_ylabel('y label');

ax.text(12, 4, 'MSDS 2022', size = 10);
ax.legend();
ax.get_children()<jupyter_output><empty_output><jupyter_text>Subplots Althought using `pyplot` seems easier, identifying the axes will come in handy once we deal with subplots or multiple plots in a figure. So far, we have just been drawing a `Figure` with only one `Axes`. But as mentioned above, `plt.subplots` allows us to actually draw a set of `Axes`. In the example below, we are making "space" for two `Axes`; `plt.subplots(1,2)` says that we draw two `Axes` using the layout `plt.subplots(nrows=1, ncols=2)`.<jupyter_code>fig, ax = plt.subplots(1,2)

# First Axes
ax[0].plot(df_ans[df_ans.dataset=='I']['x'], 
           df_ans[df_ans.dataset=='I']['y'],'r.');
ax[0].set_xlabel('x label')
ax[0].set_ylabel('y label')

# Second Axes
ax[1].plot(df_ans[df_ans.dataset=='II']['x'], 
           df_ans[df_ans.dataset=='II']['y'],'r.');
ax[1].set_xlabel('x2 label');
ax[1].set_ylabel('y2 label');
fig, ax = plt.subplots(1,2)

# First Axes
ax[0].plot(df_ans[df_ans.dataset=='I']['x'], 
           df_ans[df_ans.dataset=='I']['y'],'r.');
ax[0].set_xlabel('x label')
ax[0].set_ylabel('y label')

# Second Axes
ax[1].plot(df_ans[df_ans.dataset=='II']['x'], 
           df_ans[df_ans.dataset=='II']['y'],'r.', label='Group 2');
ax[1].set_xlabel('x2 label');
ax[1].set_ylabel('y2 label');
ax[1].plot(df_ans[df_ans.dataset=='III']['x'], 
         df_ans[df_ans.dataset=='III']["y"], 'b.', label='Group 3');
ax[1].legend(loc=4,frameon = False);

plt.tight_layout()<jupyter_output><empty_output><jupyter_text>But what if the `Axes` objects actually have the same y-axis? Can they share axes? Short answer, yes. To learn more about the sharing of axes, [read more here](https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.subplots.html).<jupyter_code>fig, ax = plt.subplots(1,2, sharey=True)

# First Axes
ax[0].plot(df_ans[df_ans.dataset=='I']['x'], 
           df_ans[df_ans.dataset=='I']['y'],'r.');
ax[0].set_xlabel('x label')
ax[0].set_ylabel('y label')

# Second Axes
ax[1].plot(df_ans[df_ans.dataset=='II']['x'], 
           df_ans[df_ans.dataset=='II']['y'],'r.', label='Group 2');
ax[1].set_xlabel('x2 label');
ax[1].plot(df_ans[df_ans.dataset=='III']['x'], 
         df_ans[df_ans.dataset=='III']["y"], 'b.', label='Group 3');
ax[1].legend(loc=4,frameon = False);

plt.tight_layout()<jupyter_output><empty_output><jupyter_text>There are other ways to create your subplots. Here, we're showing how you can use `Figure.add_subplot` and `Figure.add_axes`. Can you explore when these are more handy than `plt.subplots`?<jupyter_code>fig = plt.figure()
ax1 = fig.add_subplot(2,1,1)

## Specify location where new axes is drawn
## left, bottom, width, height
ax2 = fig.add_axes([0.5, 0.1, 0.4, 0.3])

## Plot
ax1.plot(df_ans[df_ans.dataset=='I']['x'], 
         df_ans[df_ans.dataset=='I']['y'],'r.');
ax2.plot(df_ans[df_ans.dataset=='II']['x'], 
df_ans[df_ans.dataset=='II']['y'],'b.');<jupyter_output><empty_output><jupyter_text>Below, try to change the third argument for `ax2`. See what happens using the following:- `ax2 = fig.add_subplot(2,2,4);`- `ax2 = fig.add_subplot(2,2,2);`- `ax2 = fig.add_subplot(2,2,3);`<jupyter_code>fig = plt.figure();

ax1 = fig.add_subplot(2,2,1);
ax2 = fig.add_subplot(2,2,4);

ax2.scatter(df_ans[df_ans.dataset=='II']['x'], 
           df_ans[df_ans.dataset=='II']['y'], c = 'g', label='Group 2');


# Plotting x and y for dataset group I
ax2.scatter(df_ans[df_ans.dataset=='I']['x'], 
           df_ans[df_ans.dataset=='I']['y'], c = 'r');
ax2.legend()

ax1.set_xlabel('x label');
ax1.set_ylabel('y label');
fig = plt.figure()

margs = fig.add_subplot(2,1,1)
margs.plot(df_ans[df_ans.dataset=='I']['x'], 
         df_ans[df_ans.dataset=='I']['y'],'r.');

anna = fig.add_axes([0.3, 0.1, 0.6, 0.3])
anna.plot(df_ans[df_ans.dataset=='II']['x'], 
         df_ans[df_ans.dataset=='II']['y'],'b.');

ven = fig.add_axes([.4, .3, .2, .2], facecolor='fuchsia');
ven.text(.3,.4, 'Hello');<jupyter_output><empty_output><jupyter_text>ExercisesDraw a figure with four (4) subplots. Each subplot contains the $x$-$y$ scatter plot for each group in the Anscombe's quartet. Add regression lines in each using the `np.polyfit()` function. In the fourth `Axes` or subplot, add a legend: "Empirical" and "Linear Fit". `np.polyfit()`  `fit = np.polyfit(x, y, deg=1)` (1) (2) (3) (end of exercises)  GridSpecFrom the documentation, `gridspec` "specifies the geometry of the grid that a subplot will be placed. The number of rows and number of columns of the grid need to be set. Optionally, the subplot layout parameters (e.g., left, right, etc.) can be tuned."<jupyter_code>import matplotlib.gridspec as gridspec

gs = gridspec.GridSpec(3,3)

ax1 = plt.subplot(gs[0,:])
ax2 = plt.subplot(gs[1, :-1])
ax3 = plt.subplot(gs[1:, -1])
ax4 = plt.subplot(gs[-1, 0])
ax5 = plt.subplot(gs[-1, -2])

ax1.text(0.4, 0.4, "ax1", size=20)
ax2.text(0.4, 0.4, "ax2", size=20)
ax3.text(0.4, 0.4, "ax3", size=20)
ax4.text(0.4, 0.4, "ax4", size=20)
ax5.text(0.4, 0.4, "ax5", size=20)

plt.tight_layout()
gs = gridspec.GridSpec(3,3)
ax1 = plt.subplot(gs[0,:])
ax2 = plt.subplot(gs[1, :-1])
ax3 = plt.subplot(gs[1:, -1])
ax4 = plt.subplot(gs[-1, 0])
ax5 = plt.subplot(gs[-1, -2])


fig = plt.gcf()
fig.text(0.5, 0.04, 'common X', ha='center');
fig.text(0.04, 0.5, 'common Y', va='center', rotation='vertical');
plt.tight_layout()


ax6 = fig.add_axes([.75,.2,.1,.1], facecolor="blue")
ax6.plot()
ax6.set_xticks([]);
ax6.set_yticks([]);<jupyter_output><empty_output><jupyter_text>Other Chart TypesIn this section, let's explore other chart types.<jupyter_code>doing_sci = pd.DataFrame({'response':['Bored', 'Not Great', 'OK', 
                                     'Kind of Interested', 'Excited'],
                         'pre-survey':[11, 5, 40, 25, 19], 
                         'post-survey':[12,6, 14, 30, 38]})
doing_sci.head()<jupyter_output><empty_output><jupyter_text>Bar plot<jupyter_code>xpos = np.arange(0,10,2)
width = 0.75

plt.bar(xpos, doing_sci['pre-survey'], width=width)
plt.bar(xpos+width, doing_sci['post-survey'], width=width)

plt.ylabel('% of Students')
plt.xlabel('Response')
plt.xticks(xpos + width / 2, 
           labels=doing_sci.response.to_list(), rotation=45)
plt.legend(['pre-survey', 'post-survey']);
mike = plt.gca()

mike.spines['top'].set_visible(False)
mike.spines['right'].set_visible(False)
mike.spines['bottom'].set_visible(True)
mike.spines['left'].set_visible(True);
fig, ax = plt.subplots(1)

ax.bar(xpos, doing_sci['pre-survey'], width=width)
ax.bar(xpos+width, doing_sci['post-survey'], width=width)

ax.set_ylabel('Count')
ax.set_xlabel('Classes')
ax.set_xticks(xpos + width / 2)
ax.set_xticklabels(doing_sci.response.to_list(), 
                   rotation=45)

ax.legend(['pre-survey', 'post-survey']);<jupyter_output><empty_output><jupyter_text>Pie chartAnd of course, our pie chart.<jupyter_code>fig, ax = plt.subplots(1,1,figsize=(8,8))
labels = doing_sci.response.to_list()
ax.pie(doing_sci['pre-survey'])
ax.legend(labels);
pie = plt.pie(doing_sci['pre-survey'], labels = labels, 
       autopct='%1.f%%');<jupyter_output><empty_output><jupyter_text>The fonts inside the wedges are printed in black, what if I want them in white font? How may we tweak this figure? We can take a hint by printing out the different components of the figure. Note that we "stored" the pie chart in the variable `pie`. Let's what's inside `pie` then.<jupyter_code>pie<jupyter_output><empty_output><jupyter_text>Here, we see that `pie` is a tuple. Observe the elements in the tuple above. And connect your observation with the code cell below that prints the data text (or "autotext") in white font.<jupyter_code>fig, ax = plt.subplots(1,1,figsize=(8,8))
wedges, responses, autotexts = ax.pie(doing_sci['pre-survey'], labels = labels, 
       autopct='%1.f%%', textprops=dict(color='k', size=14, ));
for autotext in autotexts:
  autotext.set_color('w');
autotexts<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15939749273082607, 'fraction_numerical': 0.020544353877687212, 'mean_word_length': 5.176037680306153, 'pattern_counts': {'":': 0, '<': 114, '<?xml version=': 0, '>': 114, 'https://': 12, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '28', 'dir': 'jupyter-structured-clean-dedup', 'id': '203067', 'license': 'MIT', 'n_tokens_mistral': 7890, 'n_tokens_neox': 7466, 'n_words': 2177, 'path': 'Intro to Data Viz with Python (Part I).ipynb', 'repo_name': 'eflegara/Data-Viz-Python', 'size': '20991'}