<jupyter_start><jupyter_text>Data Dictionary                         Dataframe                    Description                                    churn            Original dataframe created                    churn_drop            Dataframe created to drop non relevant columns                    churn_ren            Dataframe created to rename columns  Exploratory Data Analysis - EDA In order to conduct EDA first step needed is to import the required libraries:Pandas is used to clean the data, manipulate into tables and process it. It handles data load by reading from various sources; Seaborn library is used for data visualisation; Matplot is used for data visualisation; %matplotlib inline is used to format the visualisations.By importing the libraries with a short name allows us to save typing the full name of the library every time we need to call it in a function.<jupyter_code>import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np<jupyter_output><empty_output><jupyter_text>By running the code below the data from the dataset is uploaded into the dataframe using Pandas Library. A dataframe object is created transfering semi structured data (in this case a csv file) into a structured type of data.pd.read_csv works for any data file that is a csv type using Pandas library by giving an address where the data is.<jupyter_code>url = 'https://drive.google.com/file/d/1qNVNvd0N7UsbwlWpODX1-r_XB4-FWkJ-/view?usp=sharing'
path = 'https://drive.google.com/uc?export=download&id='+url.split('/')[-2]
churn = pd.read_csv(path)<jupyter_output><empty_output><jupyter_text>To check how many columns and observations are in this dataset we can use .shape function. The first number shows the number of rows or observations and the second one shows the total number of columns within the dataset. This dataset as we can see has 10,000 observations and 14 columns.<jupyter_code>churn.shape<jupyter_output><empty_output><jupyter_text>Now to read the data we need to call the dataframe created "churn" and the function .head to see what is available within this dataset. Changing the argument in between brackets allows us to add the number of rows we want to see. If we do not add a number between brackets by default the function will show the top 5 rows within the dataset (all indexes start with zero).<jupyter_code>churn.head()<jupyter_output><empty_output><jupyter_text>The .info is another function to check the data. The first column is the index for each feature we have, the second column shows the variables and the third column gives the data type, if it is an object means the data is text, float64 is a decimal number and int64 when it is whole number.It is useful information in order to start preparing the data. As per below, we can see this dataset has 3 features that are text (object) therefore qualitative data; 9 features that contain whole numbers (int64) and 2 features with decimal numbers (float64) which means quantitative data. Also shows there are no null values and gives how much memory is being used in the computer.<jupyter_code>churn.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 10000 entries, 0 to 9999
Data columns (total 14 columns):
 #   Column           Non-Null Count  Dtype  
---  ------           --------------  -----  
 0   RowNumber        10000 non-null  int64  
 1   CustomerId       10000 non-null  int64  
 2   Surname          10000 non-null  object 
 3   CreditScore      10000 non-null  int64  
 4   Geography        10000 non-null  object 
 5   Gender           10000 non-null  object 
 6   Age              10000 non-null  int64  
 7   Tenure           10000 non-null  int64  
 8   Balance          10000 non-null  float64
 9   NumOfProducts    10000 non-null  int64  
 10  HasCrCard        10000 non-null  int64  
 11  IsActiveMember   10000 non-null  int64  
 12  EstimatedSalary  10000 non-null  float64
 13  Exited           10000 non-null  int64  
dtypes: float64(2), int64(9), object(3)
memory usage: 1.1+ MB<jupyter_text>Dropping irrelevant columns As seem above this dataset contains 14 features, however not all of them are relavant to this project as not all of them affect the customer churn. Analysing the dataset we can see variables as "RowNumber", "CustomerId" and "Surname" have no effect on customer leaving the bank, threfore these 3 comuns will be removed below by using .drop function. Also a new dataframe will be created in order to keep the original "churn" dataset with the original 14 features.<jupyter_code>churn_drop = churn.drop(["RowNumber","CustomerId","Surname"],axis=1)    
churn_drop.head()<jupyter_output><empty_output><jupyter_text>All the remaining columns can influence the customer churn in some way, threfore we will keep them.  Renaming columns Few columns were renamed by using function .rename as follows.<jupyter_code>churn_ren = churn_drop.rename(columns={"CreditScore":"Credit_Score",
                                       "Balance":"Acc_Balance",
                                       "NumOfProducts":"Num_of_Products",
                                       "HasCrCard":"Credit_Card", 
                                       "IsActiveMember":"Active_Member",
                                       "EstimatedSalary":"Estimated_Salary"})<jupyter_output><empty_output><jupyter_text>By running .head again we can confirm below columns have been renamed.<jupyter_code>churn_ren.head()<jupyter_output><empty_output><jupyter_text>Identifying duplicates and missing values Identifying duplicates and missing data are important steps of data preparation to clean the data and reduce bias.  Duplicates As we can see, running the .duplicate function below the first number shows the number of duplicates equals zero and the second column shows the total number of columns equals 11, therefore there is no duplicate data within this dataframe.<jupyter_code>churn_ren[churn_ren.duplicated()]
print("Number of duplicate rows: ", churn_ren.shape)<jupyter_output>Number of duplicate rows:  (10000, 11)<jupyter_text>Missing values One of the most important goals of EDA is to identify missing values within the dataset. As previous demonstrated by the function .info this dataset has null values, however another way of exploring if the dataset has missing values is running the code below, the output false immediately tells us there is no missing data.<jupyter_code>churn_ren.isnull().values.any()<jupyter_output><empty_output><jupyter_text>We can also check the missing values for all the dataset by using .sum, it shows the variable name in the first column followed by the total number of missing values in the second column. Therefore we can confirm below there are no missing values in any feature of this dataset.<jupyter_code>churn_ren.isnull().sum()<jupyter_output><empty_output><jupyter_text>Quantitative data To discover the statistical information within the dataset we can use the function .describe. This method includes the 5 Number Summary, the summary includes the descriptive statistics of the dataset by calculating the number of entries, mean (average of values), standard deviation (average of the distance between each value and the mean), minimum value, first quartile (25% of the data), median (middle value of the data), third quartile (75% of the data) and maximum value for each one of the quantitative variables, the numerical ones. The 5 Number Summary only works for quantitative data, therefore we can note variable class is not represented below as it has qualitative data which is text.<jupyter_code>churn_ren.describe()<jupyter_output><empty_output><jupyter_text>Below the seaborn library and the function sns.boxplot were used to build a Box-Plot of the variable Ternure and Credit Score. It shows how the values are spread for this variable by demonstrating the 5 number summary: minimum score, first quartile, median, third quartile and maximum score. Also confirms the presence of outliers for the variable Credit Score.<jupyter_code>sns.boxplot(x=churn_ren["Tenure"])
sns.boxplot(x=churn_ren["Credit_Score"])<jupyter_output><empty_output><jupyter_text>Second SemesterWe can see that the Exited feature is unbalanced, with an approximation of 20.4% to the positive response and 79.6% negative, this could cause us problems in the application of our models for churn prediction.The methods and techniques that we will perform to solve this are: Resampling (Oversampling and Undersampling) and Ensembling Methods (Ensemble of Sampler).<jupyter_code>#plot the dataset after the undersampling
plt.figure(figsize=(8, 8))
sns.countplot('Exited', data=churn_ren)
plt.title('Unbalanced Classes')
plt.show()<jupyter_output>/usr/local/lib/python3.7/dist-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning<jupyter_text>1.1  Undersampling<jupyter_code># Shuffle the Dataset.
churn_shuff = churn_ren.sample(frac=1,random_state=4)

# Put all the churn class in a separate dataset.
churn_df = churn_shuff.loc[churn_shuff['Exited'] == 1]

#Randomly select 2037 observations from the non-churn (majority class)
non_churn_df = churn_shuff.loc[churn_shuff['Exited'] == 0].sample(n=2037,random_state=42)

# Concatenate both dataframes again
undersampling_df = pd.concat([churn_df, non_churn_df])

#plot the dataset after the undersampling
plt.figure(figsize=(8, 8))
sns.countplot('Exited', data=undersampling_df)
plt.title('Balanced Classes')
plt.show()<jupyter_output>/usr/local/lib/python3.7/dist-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning<jupyter_text>1.2 Oversampling (SMOTE)<jupyter_code>churn_ren["Geography"] = churn_ren["Geography"].str.replace('France', '0')
churn_ren["Geography"] = churn_ren["Geography"].str.replace('Spain', '1')
churn_ren["Geography"] = churn_ren["Geography"].str.replace('Germany', '2')
churn_ren["Geography"] = pd.to_numeric(churn_ren["Geography"])

churn_ren["Gender"] = churn_ren["Gender"].str.replace('Male', '0')
churn_ren["Gender"] = churn_ren["Gender"].str.replace('Female', '1')

churn_ren["Gender"] = pd.to_numeric(churn_ren["Gender"])
churn_ren.head()
from imblearn.over_sampling import SMOTE

# Resample the minority class. You can change the strategy to 'auto' if you are not sure.
sm = SMOTE(sampling_strategy='minority', random_state=7)

# Fit the model to generate the data.
oversampled_trainX, oversampled_trainY = sm.fit_resample(churn_ren.drop('Exited', axis=1), churn_ren['Exited'])
oversampled_train = pd.concat([pd.DataFrame(oversampled_trainY), pd.DataFrame(oversampled_trainX)], axis=1)<jupyter_output><empty_output><jupyter_text><jupyter_code>#plot the dataset after the undersampling
plt.figure(figsize=(8, 8))
sns.countplot('Exited', data=oversampled_train)
plt.title('Balanced Classes')
plt.show()<jupyter_output>/usr/local/lib/python3.7/dist-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning<jupyter_text>2 Ensembling Methods (Ensemble of Sampler)<jupyter_code>from imblearn.ensemble import BalancedBaggingClassifier
from sklearn.tree import DecisionTreeClassifier

#Create an object of the classifier.
bbc = BalancedBaggingClassifier(base_estimator=DecisionTreeClassifier(),
                                sampling_strategy='auto',
                                replacement=False,
                                random_state=0,
                                max_samples=1.0, max_features=1.0, bootstrap=True, bootstrap_features=False, oob_score=False, warm_start=False, n_jobs=None, verbose=0, sampler=None)

y_train = churn_ren['Exited']
X_train = churn_ren.drop(['Exited'], axis=1, inplace=False)

#Train the classifier.
bbc.fit(X_train, y_train)
preds = bbc.predict(X_test)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07380490956072351, 'fraction_numerical': 0.01978359173126615, 'mean_word_length': 4.3522039757994815, 'pattern_counts': {'":': 6, '<': 69, '<?xml version=': 0, '>': 69, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '189972', 'license': 'MIT', 'n_tokens_mistral': 3834, 'n_tokens_neox': 3449, 'n_words': 1347, 'path': 'Colab_Strategic_Thinking_Final.ipynb', 'repo_name': 'scspx/Bank-customer-churn-prediction', 'size': '12384'}