<jupyter_start><jupyter_text>This module focuses on Basic Visualization Tools<jupyter_code>UserPath= "/home/cerbero/"

ImportPath = UserPath+"Documents/IBM DV0101EN/01/"
ExportPath = UserPath+"Documents/IBM DV0101EN/02/"

import pandas as pd
df_canada = pd.read_excel(ImportPath+"Canada Immigration Data.xlsx")
df_canada.describe()
df_canada.head()
df_canada.shape
df_01 = df_canada.sort_values(["Total"], ascending = False, axis = 0)
df_01 = df_01.set_index("Country")
df_01.head()
df_02 = df_01.head()
df_02 = df_02.drop(['Continent','Region','DevName','Total'], axis=1)
df_02 = df_02.transpose()
df_02
import matplotlib as mpl
import matplotlib.pyplot as plt
%matplotlib inline
df_02.plot(kind="area", alpha=0.8, stacked=True, figsize=(14,8)) 
#stacked=False makes plot translucent, doesn't work here 
#The unstacked plot has a default transparency (alpha value) at 0.5. 
#We can modify this value by passing in the alpha parameter

plt.title("Immigration to Canada - Top 5 Trend")
plt.ylabel("People")
plt.xlabel("Years")

# plt.show()  #works without this line. Not sure what it is for...
# the show() above messes up with the savefig function

plt.savefig(ExportPath+"Immigration to Canada - Top 5 Trend.png")
years = list(map(int, range(1980, 2014)))
print(years)
# get the 4 countries with the least contribution
# Lab wanted to use 5 countries, but it does not look good - Vietnam is much larger

# Will use the Artist layer to show an alternative way to plot
df_least4 = df_canada.tail(4)


df_least4 = df_least4[years].transpose() 
df_least4.head()

ax = df_least4.plot(kind='area', alpha=0.55, stacked=True, figsize=(14, 8)) #object-oriented

ax.set_title('Immigration Trend of 4 Countries with Least Contribution to Immigration') #notice the setters
ax.set_ylabel('Number of Immigrants')
ax.set_xlabel('Years')

plt.savefig(ExportPath+"Immigration to Canada - Bottom 4 Trend.png")<jupyter_output><empty_output><jupyter_text>You can use an Axes instance of your current plot and store it in a variable (eg. ax). You can add more elements by calling methods with a little change in syntax (by adding "set_" to the previous methods). For example, use ax.set_title() instead of plt.title() to add title, or ax.set_xlabel() instead of plt.xlabel() to add label to the x-axis.This option sometimes is more transparent and flexible to use for advanced plots (in particular when having multiple plots, as you will see later).<jupyter_code>df_least4 #Western Sahara is there, but barely
#might as well drop the index. Not very useful.
df_canada = df_canada.set_index("Country")
df_canada
df_canada[2013].plot(kind="hist", figsize=(20,10))

plt.title("Histogram of Immigration to Canada from 195 countries in 2013")
plt.ylabel("Number of Countries")
plt.xlabel("Partitions of Number of Immigrants")

plt.savefig(ExportPath+"Histogram of 195 countries.png")<jupyter_output><empty_output><jupyter_text>The histogram above is not very effective. Letś fix it.<jupyter_code>import numpy as np
count, bin_edges = np.histogram(df_canada[2013]) #this numpy function gives nicer bin-edges; count is unused
# default is 10 bins

df_canada[2013].plot(kind = "hist", xticks = bin_edges, figsize=(14,8)) #xticks now is bin_edges

# plt.grid(zorder = 5)
# plt.bar(bin_edges, 195, zorder = 0) #z-order did not work well
plt.grid()
plt.title("Histogram of Immigration to Canada from 195 countries in 2013")
plt.ylabel("Number of Countries")
plt.xlabel("Partitions of Number of Immigrants")

plt.savefig(ExportPath+"Improved Histogram of 195 countries.png")
print(count, bin_edges) #two arrays are returned, naturally
df_canada.loc[['Denmark', 'Norway', 'Sweden'], years]
df_canada.loc[['Denmark', 'Norway', 'Sweden'], years].plot.hist()
#need to transpose
df_t = df_canada.loc[['Denmark', 'Norway', 'Sweden'], years].transpose()
df_t.head()
df_t.plot(kind='hist', figsize=(10, 6))

plt.title('Histogram of Immigration from Denmark, Norway, and Sweden from 1980 - 2013')
plt.ylabel('Number of Years')
plt.xlabel('Number of Immigrants')

#still does not look good
# let's get the x-tick values
count, bin_edges = np.histogram(df_t, 15)

# un-stacked histogram
df_t.plot(kind ='hist', 
          figsize=(10, 6),
          bins=15,
          alpha=0.6,
          xticks=bin_edges,
          color=['coral', 'darkslateblue', 'mediumseagreen']
         )

plt.title('Histogram of Immigration from Denmark, Norway, and Sweden from 1980 - 2013')
plt.ylabel('Number of Years')
plt.xlabel('Number of Immigrants')
# Tip: For a full listing of colors available in Matplotlib, run the following code in your python shell:


import matplotlib
for name, hex in matplotlib.colors.cnames.items():
    print(name, hex)<jupyter_output>aliceblue #F0F8FF
antiquewhite #FAEBD7
aqua #00FFFF
aquamarine #7FFFD4
azure #F0FFFF
beige #F5F5DC
bisque #FFE4C4
black #000000
blanchedalmond #FFEBCD
blue #0000FF
blueviolet #8A2BE2
brown #A52A2A
burlywood #DEB887
cadetblue #5F9EA0
chartreuse #7FFF00
chocolate #D2691E
coral #FF7F50
cornflowerblue #6495ED
cornsilk #FFF8DC
crimson #DC143C
cyan #00FFFF
darkblue #00008B
darkcyan #008B8B
darkgoldenrod #B8860B
darkgray #A9A9A9
darkgreen #006400
darkgrey #A9A9A9
darkkhaki #BDB76B
darkmagenta #8B008B
darkolivegreen #556B2F
darkorange #FF8C00
darkorchid #9932CC
darkred #8B0000
darksalmon #E9967A
darkseagreen #8FBC8F
darkslateblue #483D8B
darkslategray #2F4F4F
darkslategrey #2F4F4F
darkturquoise #00CED1
darkviolet #9400D3
deeppink #FF1493
deepskyblue #00BFFF
dimgray #696969
dimgrey #696969
dodgerblue #1E90FF
firebrick #B22222
floralwhite #FFFAF0
forestgreen #228B22
fuchsia #FF00FF
gainsboro #DCDCDC
ghostwhite #F8F8FF
gold #FFD700
goldenrod #DAA520
gray #808080
green #008000
greenyellow #ADFF2F
[...]<jupyter_text>Finally, the bar chart<jupyter_code>df_iceland = df_canada.loc["Iceland", years]
df_iceland.head(), df_iceland.tail()
df_iceland.plot(kind="bar", figsize =(14,8))

plt.title("Immigration from Iceland to Canada over the Years")
plt.xlabel("Years")
plt.ylabel("Number of Immigrants")

plt.savefig(ExportPath+"Immigration from Iceland to Canada over the Years - Bar Chart.png")
df_iceland.plot(kind='bar', figsize=(10, 6), rot=90) 

plt.xlabel('Year')
plt.ylabel('Number of Immigrants')
plt.title('Icelandic Immigrants to Canada from 1980 to 2013')

# Annotate arrow
plt.annotate('',                      # s: str. will leave it blank for no text
             xy=(32, 70),             # place head of the arrow at point (year 2012 , pop 70)
             xytext=(28, 20),         # place base of the arrow at point (year 2008 , pop 20)
             xycoords='data',         # will use the coordinate system of the object being annotated 
             arrowprops=dict(arrowstyle='->', connectionstyle='arc3', color='blue', lw=2)
            )

# Annotate Text
plt.annotate('2008 - 2011 Financial Crisis', # text to display
             xy=(28, 30),                    # start the text at at point (year 2008 , pop 30)
             rotation=72.5,                  # based on trial and error to match the arrow
             va='bottom',                    # want the text to be vertically 'bottom' aligned
             ha='left',                      # want the text to be horizontally 'left' algned.
            )

plt.savefig(ExportPath+"Immigration from Iceland to Canada over the Years - Bar Chart with Arrow.png")
#Using horizontal bars
# sort dataframe on 'Total' column (descending)
df_top15 = df_canada.sort_values(by='Total', ascending=True)

# get top 15 countries
df_top15 = df_top15['Total'].tail(15)
df_top15
# generate plot
df_top15.plot(kind='barh', figsize=(12, 12), color='steelblue')  #kind="barh" is key here
plt.xlabel('Number of Immigrants')
plt.title('Top 15 Countries in Immigration to Canada between 1980 - 2013')

# annotate value labels to each country
for index, value in enumerate(df_top15): 
    label = format(int(value), ',') # format int with commas
    
    # place text at the end of bar (subtracting 47000 from x, and 0.1 from y to make it fit within the bar)
    plt.annotate(label, xy=(value - 47000, index - 0.10), color='white')
    
plt.savefig(ExportPath+"Top 15 Countries in Immigration to Canada between 1980 - 2013.png")
df_canada.to_excel(ExportPath+"Canada Immigration Data.xlsx")
df_iceland.to_excel(ExportPath+"Immigration from Iceland to Canada.xlsx")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1080757248981548, 'fraction_numerical': 0.05271986580397795, 'mean_word_length': 4.609543010752688, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '497305', 'license': 'MIT', 'n_tokens_mistral': 3200, 'n_tokens_neox': 2789, 'n_words': 884, 'path': '02 Second Module.ipynb', 'repo_name': 'Gus-Viola/IBM-DV0101EN', 'size': '8346'}