<jupyter_start><jupyter_text>Pandas InstructionsThis assignment will be done completely inside this Jupyter notebook with answers placed in the cell provided.All python imports that are needed shown.Follow all the instructions in this notebook to complete these tasks.    Make sure the CSV data files is in the same folder as this notebook - alumni.csv<jupyter_code>#!pip install matplotlib
# Imports needed to complete this exam

import pandas as pd
import matplotlib.pyplot as plt<jupyter_output>In C:\Users\user\Anaconda3\lib\site-packages\matplotlib\mpl-data\stylelib\_classic_test.mplstyle: 
The text.latex.preview rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In C:\Users\user\Anaconda3\lib\site-packages\matplotlib\mpl-data\stylelib\_classic_test.mplstyle: 
The mathtext.fallback_to_cm rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In C:\Users\user\Anaconda3\lib\site-packages\matplotlib\mpl-data\stylelib\_classic_test.mplstyle: Support for setting the 'mathtext.fallback_to_cm' rcParam is deprecated since 3.3 and will be removed two minor releases later; use 'mathtext.fallback : 'cm' instead.
In C:\Users\user\Anaconda3\lib\site-packages\matplotlib\mpl-data\stylelib\_classic_test.mplstyle: 
The validate_bool_maybe_none function was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In C:\Users\user\Anaconda3\lib\site-packages\matplotlib\mpl-data\stylelib\_classic_test.mplstyle[...]<jupyter_text>Question 1 :  Import CSV file (1 Mark)Write code to load the alumni csv dataset into a Pandas DataFrame called 'alumni'.<jupyter_code>#loading the dataset
alumni = pd.read_csv('alumni.csv')<jupyter_output><empty_output><jupyter_text>Question 2 :  Understand the data set (5 Marks)Use the following pandas commands to understand the data set: a) head, b) tail, c) dtypes, d) info, e) describe<jupyter_code>#a) (1)
# checking the first 5 rows
alumni.head()
#b) (1) # The last 5 columns
alumni.tail()
#c) (1)# checking the data types
alumni.dtypes
#d) (1) # general info on the data types
alumni.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 88 entries, 0 to 87
Data columns (total 8 columns):
Year Graduated    88 non-null int64
Gender            88 non-null object
Marital Status    88 non-null object
Diploma Type      88 non-null object
Defaulted         88 non-null object
Salary            88 non-null int64
Fee               88 non-null int64
Savings ($)       88 non-null object
dtypes: int64(3), object(5)
memory usage: 5.6+ KB<jupyter_text>The above shows that there are no null values in the dataset<jupyter_code>#e) (1)
alumni.describe()<jupyter_output><empty_output><jupyter_text>Question 3 :  Cleaning the data set - part A (3 Marks)a) Use clean_currency method below to strip out commas and dollar signs from Savings ($) column and put into a new column called 'Savings'.<jupyter_code>def clean_currency(curr):
    return float(curr.replace(",", "").replace("$", ""))

clean_currency("$66,000")
 
#using the clean_currency method using the .apply method
alumni['Savings'] = alumni['Savings ($)'].apply(clean_currency)
alumni.head()<jupyter_output><empty_output><jupyter_text>b) Uncomment 'alumni.dtypes.Savings' to check that the type change has occurred<jupyter_code>#b) (1)
alumni.dtypes.Savings<jupyter_output><empty_output><jupyter_text>Question 4 :  Cleaning the data set - part B (5 Marks)a) Run the 'alumni["Gender"].value_counts()' to see the incorrect 'M' fields that need to be converted to 'Male'<jupyter_code># a) (1)
alumni['Gender'].value_counts()<jupyter_output><empty_output><jupyter_text>b) Now use a '.str.replace' on the 'Gender' column to covert the incorrect 'M' fields. Hint: We must use ^...$ to restrict the pattern to match the whole string.<jupyter_code>#method 2
alumni['Gender'].str.replace('^[M]$','Male')
# b) (1)
import re
def gender_clean(var):
    return re.sub('^M$','Male',var)
alumni['Gender'].apply(gender_clean)<jupyter_output><empty_output><jupyter_text>c) That didn't the set alumni["Gender"] column however. You will need to update the column when using the replace command 'alumni["Gender"]=', show how this is done below<jupyter_code># c) (1)
alumni["Gender"]=alumni['Gender'].apply(gender_clean)<jupyter_output><empty_output><jupyter_text>d) You can set it directly by using the df.loc command, show how this can be done by using the 'df.loc[row_indexer,col_indexer] = value' command to convert the 'M' to 'Male'<jupyter_code># d) (1)
alumni.loc[alumni['Gender'] == 'M'] = 'Male'<jupyter_output><empty_output><jupyter_text>e) Now run the 'value_counts' for Gender again to see the correct columns - 'Male' and 'Female'<jupyter_code># e) (1)
alumni['Gender'].value_counts()<jupyter_output><empty_output><jupyter_text>Question 5 :  Working with the data set (4)a) get the median, b) mean and c) standard deviation for the 'Salary' column<jupyter_code>alumni.head(2)
# a)(1)
print('The median is:',round(alumni['Salary'].median(),2))
# b)(1)
print('The mean is:',round(alumni['Salary'].mean(),2))
# c)(1)
print('The mean is:',round(alumni['Salary'].std(),2))<jupyter_output>The mean is: 21234.13<jupyter_text>d) identify which alumni paid more than $15000 in fees, using the 'Fee' column<jupyter_code># d) (1)
alumni[alumni['Fee'] > 15000]<jupyter_output><empty_output><jupyter_text>Question 6 :  Visualise the data set (4 Marks)a) Using the 'Diploma Type' column, plot a bar chart and show its value counts.<jupyter_code>#a) (1)
alumni['Diploma Type'].value_counts().plot.bar();<jupyter_output><empty_output><jupyter_text>b) Now create a box plot comparison between 'Savings' and 'Salary' columns<jupyter_code>#b) (1)
import seaborn as sns
savings_salary = ["Savings", "Salary"]
sns.catplot(data=alumni[savings_salary],kind = 'box');<jupyter_output><empty_output><jupyter_text>c) Generate a histogram with the 'Salary' column and use 12 bins.<jupyter_code>#c) (1)
sns.distplot(alumni['Salary'],bins = 12,kde = False)<jupyter_output><empty_output><jupyter_text>d) Generate a scatter plot comparing 'Salary' and 'Savings' columns.<jupyter_code>#d) (1)
sns.scatterplot(x = 'Salary',y = 'Savings', data = alumni);<jupyter_output><empty_output><jupyter_text>Question 7 :  Contingency Table (2 Marks)Using both the 'Martial Status' and 'Defaulted' create a contingency table. Hint: crosstab<jupyter_code># Q7 (2)
category = pd.crosstab(alumni['Marital Status'],alumni['Defaulted'])
category
#stacked bar plot to show propotion of approved and unapproved loans.
category.div(category.sum(1).astype(float),axis = 0).plot(kind = 'bar',stacked=True,figsize = (4,4));<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12225048630854407, 'fraction_numerical': 0.01780637438276223, 'mean_word_length': 6.088016967126193, 'pattern_counts': {'":': 0, '<': 71, '<?xml version=': 0, '>': 72, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '192187', 'license': 'MIT', 'n_tokens_mistral': 2439, 'n_tokens_neox': 2263, 'n_words': 661, 'path': 'Pandas Analysis Assignment 1/Pandas Analysis Assignment 1.ipynb', 'repo_name': 'DennisChangach/Africa-Data-School-Projects-and-Assignments', 'size': '6683'}