<jupyter_start><jupyter_text>**Project 1 - Ensemble Learning**  **Data Description:**   
The data is related with direct marketing campaigns of a  Portuguese banking institution. The marketing campaigns  were based on phone calls. Often, more than one contact to  the same client was required, in order to access if the product  (bank term deposit) would be ('yes') or not ('no') subscribed.   

 **Domain:**   
  Banking  

 **Context:**  
Leveraging customer information is paramount for most  businesses. In the case of a bank, attributes of customers like  the ones mentioned below can be crucial in strategizing a  marketing campaign when launching a new product.  

 **Attribute Information:** 

* age (numeric)  
* job : type of job (categorical:  'admin.','blue-collar','entrepreneur','housemaid','manage ment','retired','self-employed','services','student','technicia n','unemployed','unknown')  
* marital : marital status (categorical:  'divorced','married','single','unknown'; note: 'divorced'  means divorced or widowed)  
* education (categorical: 'basic.4y','basic.6y','basic.9y','high.school','illiterate','profes sional.course','university.degree','unknown') 
* default: has credit in default? (categorical:  'no','yes','unknown')  
* balance: average yearly balance, in euros (numeric)  
* housing: has housing loan? (categorical:  'no','yes','unknown')  
* loan: has personal loan? (categorical: 'no','yes','unknown')  
* contact: contact communication type (categorical:  'cellular','telephone')  
* day: last contact day of the month (numeric 1 -31)  
* month: last contact month of year (categorical: 'jan', 'feb',  'mar', ..., 'nov', 'dec')  
*  duration: last contact duration, in seconds (numeric).  Important note: this attribute highly affects the output  target (e.g., if duration=0 then y='no'). Yet, the duration is  not known before a call is performed. Also, after the end  of the call y is obviously known. Thus, this input should  only be included for benchmark purposes and should be  discarded if the intention is to have a realistic predictive  model.  
*  campaign: number of contacts performed during this  campaign and for this client (numeric, includes last  contact)  
*  pdays: number of days that passed by after the client  was last contacted from a previous campaign (numeric;  999 means client was not previously contacted)  
* previous: number of contacts performed before this  campaign and for this client (numeric)  
*  poutcome: outcome of the previous marketing  campaign (categorical: 'failure','nonexistent','success')  
* target: has the client subscribed a term deposit? (binary:  "yes","no") 


 **Objective**
 **The classification goal is to predict if the client will subscribe  (yes/no) a term deposit (variable y).**    **Learning Outcomes:**  
 * Exploratory Data Analysis  
 * Preparing the data to train a model  
 * Training and making predictions using an Ensemble  Model  
 * Tuning an Ensemble model    
 **Importing all the required libraries**<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from scipy.stats import pearsonr

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler,MinMaxScaler,RobustScaler
from sklearn.linear_model import LogisticRegression
from sklearn.naive_bayes import GaussianNB,MultinomialNB,BernoulliNB
from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC
from sklearn.tree import DecisionTreeClassifier

from IPython.display import Image
from sklearn import tree
from os import system

from sklearn.ensemble import BaggingClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.ensemble import ExtraTreesClassifier

from sklearn.ensemble import AdaBoostClassifier
from sklearn.ensemble import GradientBoostingClassifier

from sklearn.metrics import accuracy_score,confusion_matrix,classification_report
from sklearn.metrics import precision_score,recall_score,roc_auc_score

from google.colab import files
import io<jupyter_output><empty_output><jupyter_text>**Importing the Dataset to Notebook.**<jupyter_code>path = '/content/drive/MyDrive/Colab Notebooks/AIML/bank-full - Govindaraj V.csv'
data = pd.read_csv(path)

data.head() #Looking for 1st 5 rows of the dataset
data.tail() #Looking for last 5 rows of the dataset
data.shape #looking for the shape of the dataset<jupyter_output><empty_output><jupyter_text>**Performing Exploratory Data Analysis**<jupyter_code>data.dtypes.to_frame('Data Types')
data.info()<jupyter_output><empty_output><jupyter_text>**Checking for missing values**<jupyter_code>data.isna().sum()
data.head(1)
sns.countplot(data['poutcome'])
sns.countplot(data['contact'])
sns.countplot(data['education'])
sns.countplot(data['job'])
data['job'].value_counts()
data.describe()<jupyter_output><empty_output><jupyter_text>**5 Point Summary understanding** 1. Outliers are present in age,balance,duration,campaign,pdays,previous columns

2. balance,duration,campaign,pdays,previous are right skewed
   **Distribution of Numerical features**<jupyter_code>fig,axe=plt.subplots(nrows=4,ncols=2,figsize=(10,15))
axe=axe.flatten()
sns.distplot(data['age'],ax=axe[0])
sns.distplot(data['balance'],ax=axe[1])
sns.distplot(data['day'],ax=axe[2])
sns.distplot(data['duration'],ax=axe[3])
sns.distplot(data['campaign'],ax=axe[4])
sns.distplot(data['pdays'],ax=axe[5])
sns.distplot(data['previous'],ax=axe[6])

fig.tight_layout()
fig.show()

data.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 45211 entries, 0 to 45210
Data columns (total 17 columns):
 #   Column     Non-Null Count  Dtype 
---  ------     --------------  ----- 
 0   age        45211 non-null  int64 
 1   job        45211 non-null  object
 2   marital    45211 non-null  object
 3   education  45211 non-null  object
 4   default    45211 non-null  object
 5   balance    45211 non-null  int64 
 6   housing    45211 non-null  object
 7   loan       45211 non-null  object
 8   contact    45211 non-null  object
 9   day        45211 non-null  int64 
 10  month      45211 non-null  object
 11  duration   45211 non-null  int64 
 12  campaign   45211 non-null  int64 
 13  pdays      45211 non-null  int64 
 14  previous   45211 non-null  int64 
 15  poutcome   45211 non-null  object
 16  Target     45211 non-null  object
dtypes: int64(7), object(10)
memory usage: 5.9+ MB<jupyter_text>**Distribution of Categorical features**<jupyter_code>fig,axe=plt.subplots(nrows=5,ncols=2,figsize=(10,20))
axe=axe.flatten()
plot=sns.countplot(x=data['job'],data=data,ax=axe[0])
plt.setp(plot.get_xticklabels(),rotation=90)
sns.countplot(x=data['marital'],data=data,ax=axe[1])

sns.countplot(x=data['education'],data=data,ax=axe[2])
sns.countplot(x=data['default'],data=data,ax=axe[3])
sns.countplot(x=data['housing'],data=data,ax=axe[4])
sns.countplot(x=data['loan'],data=data,ax=axe[5])
sns.countplot(x=data['contact'],data=data,ax=axe[6])
sns.countplot(x=data['month'],data=data,ax=axe[7])
sns.countplot(x=data['poutcome'],data=data,ax=axe[8])
sns.countplot(x=data['Target'],data=data,ax=axe[9])

fig.tight_layout()
fig.show()<jupyter_output><empty_output><jupyter_text>**Measure the skewness of numerical feature**<jupyter_code>data.skew().to_frame('Skewness').T<jupyter_output><empty_output><jupyter_text>**Checking the Outliers**<jupyter_code>fig,axe=plt.subplots(nrows=4,ncols=2,figsize=(10,20))
axe=axe.flatten()
sns.boxplot(x='age',data=data,ax=axe[0])
sns.boxplot(x='balance',data=data,ax=axe[1])
sns.boxplot(x='day',data=data,ax=axe[2])
sns.boxplot(x='duration',data=data,ax=axe[3])
sns.boxplot(x='campaign',data=data,ax=axe[4])
sns.boxplot(x='pdays',data=data,ax=axe[5])
sns.boxplot(data['previous'],ax=axe[6])
fig.tight_layout()
fig.show()
axe.flat[-1].set_visible(False)<jupyter_output>/usr/local/lib/python3.6/dist-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning<jupyter_text>**Label Encoding for string feature**<jupyter_code>from sklearn.preprocessing import LabelEncoder
df=data.copy()
encoder=LabelEncoder()

df['default']=encoder.fit_transform(df['default']).astype('int')

df['marital']=encoder.fit_transform(df['marital']).astype('int')
df['loan']=encoder.fit_transform(df['loan']).astype('int')
df['housing']=encoder.fit_transform(df['housing']).astype('int')
df['Target']=encoder.fit_transform(df['Target']).astype('int')
df['month']=encoder.fit_transform(df['month']).astype('int')
df['poutcome']=encoder.fit_transform(df['poutcome']).astype('int')
df['education']=encoder.fit_transform(df['education']).astype('int')

df['job']=encoder.fit_transform(df['job']).astype('int')
df['contact']=encoder.fit_transform(df['contact']).astype('int')


df.head(2)
df['job']=encoder.fit_transform(df['job']).astype('int')
df['contact']=encoder.fit_transform(df['contact']).astype('int')

df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 45211 entries, 0 to 45210
Data columns (total 17 columns):
 #   Column     Non-Null Count  Dtype
---  ------     --------------  -----
 0   age        45211 non-null  int64
 1   job        45211 non-null  int64
 2   marital    45211 non-null  int64
 3   education  45211 non-null  int64
 4   default    45211 non-null  int64
 5   balance    45211 non-null  int64
 6   housing    45211 non-null  int64
 7   loan       45211 non-null  int64
 8   contact    45211 non-null  int64
 9   day        45211 non-null  int64
 10  month      45211 non-null  int64
 11  duration   45211 non-null  int64
 12  campaign   45211 non-null  int64
 13  pdays      45211 non-null  int64
 14  previous   45211 non-null  int64
 15  poutcome   45211 non-null  int64
 16  Target     45211 non-null  int64
dtypes: int64(17)
memory usage: 5.9 MB<jupyter_text>**Build the Base classification ML Model**<jupyter_code>df.head(1)
x=df.drop('Target',axis=1)
y=df['Target']
x.head()<jupyter_output><empty_output><jupyter_text>**Split the Records**<jupyter_code>xtrain,xtest,ytrain,ytest=train_test_split(x,y,test_size=.3,random_state=3)<jupyter_output><empty_output><jupyter_text>**Scale the data**<jupyter_code>scaler=StandardScaler()
xtrain_scaled=scaler.fit_transform(xtrain)
xtest_scaled=scaler.fit_transform(xtest)<jupyter_output><empty_output><jupyter_text>**Logistic Regression Model**<jupyter_code>log_reg=LogisticRegression(solver='liblinear')
log_reg.fit(xtrain,ytrain)
ypred=log_reg.predict(xtest)
print('\nThe Accuracy score\n',accuracy_score(ytest,ypred))
print('\n The Confusion matrix \n',confusion_matrix(ytest,ypred))
print('\n The Classification Report \n',classification_report(ytest,ypred))<jupyter_output>The Accuracy score
 0.8923621350634031

 The Confusion matrix 
 [[11751   218]
 [ 1242   353]]

 The Classification Report 
               precision    recall  f1-score   support

           0       0.90      0.98      0.94     11969
           1       0.62      0.22      0.33      1595

    accuracy                           0.89     13564
   macro avg       0.76      0.60      0.63     13564
weighted avg       0.87      0.89      0.87     13564<jupyter_text>**Naive Bayes**<jupyter_code>naive=GaussianNB()
naive.fit(xtrain,ytrain)
ypred_naive=naive.predict(xtest)
print('\nThe Accuracy score\n',accuracy_score(ytest,ypred_naive))
print('\n The Confusion matrix \n',confusion_matrix(ytest,ypred_naive))
print('\n The Classification Report \n',classification_report(ytest,ypred_naive))<jupyter_output><empty_output><jupyter_text>**SVC**<jupyter_code>svc=SVC()
svc.fit(xtrain,ytrain)
ypred_svc=svc.predict(xtest)
print('\nThe Accuracy score\n',accuracy_score(ytest,ypred_svc))
print('\n The Confusion matrix \n',confusion_matrix(ytest,ypred_svc))
print('\n The Classification Report \n',classification_report(ytest,ypred_svc))<jupyter_output><empty_output><jupyter_text>**Decision Tree Classifier**<jupyter_code>dtree=DecisionTreeClassifier()
dtree.fit(xtrain,ytrain)
ypred_dtree=dtree.predict(xtest)
print('\nThe Accuracy score\n',accuracy_score(ytest,ypred_dtree))
print('\n The Confusion matrix \n',confusion_matrix(ytest,ypred_dtree))
print('\n The Classification Report \n',classification_report(ytest,ypred_dtree))<jupyter_output><empty_output><jupyter_text>**Ensemble Techniques**  **1. Bagging Classifier**<jupyter_code>bagc=BaggingClassifier(base_estimator=dtree,n_estimators=200,random_state=1)
bagc.fit(xtrain,ytrain)
ypred_bagc=bagc.predict(xtest)
print('\nThe Accuracy score\n',accuracy_score(ytest,ypred_bagc))
print('\n The Confusion matrix \n',confusion_matrix(ytest,ypred_bagc))
print('\n The Classification Report \n',classification_report(ytest,ypred_bagc))<jupyter_output><empty_output><jupyter_text>**Randomforest**<jupyter_code>randf=RandomForestClassifier(n_estimators=200,random_state=2,max_features=4)
randf.fit(xtrain,ytrain)
ypred_rand=randf.predict(xtest)

print('\nThe Accuracy score\n',accuracy_score(ytest,ypred_rand))
print('\n The Confusion matrix \n',confusion_matrix(ytest,ypred_rand))
print('\n The Classification Report \n',classification_report(ytest,ypred_rand))<jupyter_output><empty_output><jupyter_text>**Adaboost**<jupyter_code>adab=AdaBoostClassifier(n_estimators=200,random_state=2)
adab.fit(xtrain,ytrain)
ypred_ada=adab.predict(xtest)

print('\nThe Accuracy score\n',accuracy_score(ytest,ypred_ada))
print('\n The Confusion matrix \n',confusion_matrix(ytest,ypred_ada))
print('\n The Classification Report \n',classification_report(ytest,ypred_ada))<jupyter_output><empty_output><jupyter_text>**Gradient Boosting**<jupyter_code>gb=GradientBoostingClassifier(n_estimators=200,random_state=2)
gb.fit(xtrain,ytrain)
ypred_gb=gb.predict(xtest)

print('\nThe Accuracy score\n',accuracy_score(ytest,ypred_gb))
print('\n The Confusion matrix \n',confusion_matrix(ytest,ypred_gb))
print('\n The Classification Report \n',classification_report(ytest,ypred_gb))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13025742017161346, 'fraction_numerical': 0.034885356590237726, 'mean_word_length': 5.585919407132932, 'pattern_counts': {'":': 0, '<': 79, '<?xml version=': 0, '>': 79, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '20', 'dir': 'jupyter-structured-clean-dedup', 'id': '644700', 'license': 'MIT', 'n_tokens_mistral': 5287, 'n_tokens_neox': 4962, 'n_words': 1031, 'path': 'Project_1_Ensemble_techniques.ipynb', 'repo_name': 'Nishadansh47/AIML', 'size': '14218'}