<jupyter_start><jupyter_text>**Problem Statement** Our client is an Insurance company that has provided Health Insurance to its customers now they need your help in building a model to predict whether the policyholders (customers) from past year will also be interested in Vehicle Insurance provided by the company.An insurance policy is an arrangement by which a company undertakes to provide a guarantee of compensation for specified loss, damage, illness, or death in return for the payment of a specified premium. A premium is a sum of money that the customer needs to pay regularly to an insurance company for this guarantee.For example, you may pay a premium of Rs. 5000 each year for a health insurance cover of Rs. 200,000/- so that if, God forbid, you fall ill and need to be hospitalised in that year, the insurance provider company will bear the cost of hospitalisation etc. for upto Rs. 200,000. Now if you are wondering how can company bear such high hospitalisation cost when it charges a premium of only Rs. 5000/-, that is where the concept of probabilities comes in picture. For example, like you, there may be 100 customers who would be paying a premium of Rs. 5000 every year, but only a few of them (say 2-3) would get hospitalised that year and not everyone. This way everyone shares the risk of everyone else.Just like medical insurance, there is vehicle insurance where every year customer needs to pay a premium of certain amount to insurance provider company so that in case of unfortunate accident by the vehicle, the insurance provider company will provide a compensation (called ‘sum assured’) to the customer.Building a model to predict whether a customer would be interested in Vehicle Insurance is extremely helpful for the company because it can then accordingly plan its communication strategy to reach out to those customers and optimise its business model and revenue.Now, in order to predict, whether the customer would be interested in Vehicle insurance, you have information about demographics (gender, age, region code type), Vehicles (Vehicle Age, Damage), Policy (Premium, sourcing channel) etc.  **Attribute Information** 1. id :	Unique ID for the customer2. Gender	: Gender of the customer3. Age :	Age of the customer4. Driving_License	0 : Customer does not have DL, 1 : Customer already has DL5. Region_Code :	Unique code for the region of the customer6. Previously_Insured	: 1 : Customer already has Vehicle Insurance, 0 : Customer doesn't have Vehicle Insurance7. Vehicle_Age :	Age of the Vehicle8. Vehicle_Damage	 :1 : Customer got his/her vehicle damaged in the past. 0 : Customer didn't get his/her vehicle damaged in the past.9. Annual_Premium	: The amount customer needs to pay as premium in the year10. PolicySalesChannel :	Anonymized Code for the channel of outreaching to the customer ie. Different Agents, Over Mail, Over Phone, In Person, etc.11. Vintage :	Number of Days, Customer has been associated with the company12. Response :	1 : Customer is interested, 0 : Customer is not interested 1. **Prepare** **the** **problem**     Importing Required Libraries<jupyter_code># import libraries
import pandas  as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.preprocessing import LabelEncoder 
from sklearn.model_selection import train_test_split

from imblearn.over_sampling import RandomOverSampler
from collections import Counter
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import precision_score,recall_score,accuracy_score,f1_score,confusion_matrix,roc_auc_score,classification_report
from sklearn.ensemble import RandomForestClassifier
from xgboost import XGBClassifier

sns.set_theme(style="darkgrid")<jupyter_output><empty_output><jupyter_text>**Load** **Dataset**<jupyter_code>from google.colab import drive
drive.mount('/content/drive')
data=pd.read_csv('/content/drive/MyDrive/csv files/TRAIN-HEALTH INSURANCE CROSS SELL PREDICTION.csv')<jupyter_output><empty_output><jupyter_text>**2**. **Summarize** **Data**<jupyter_code>data.shape
data.head(10)
data.tail()
print(data.info())
print()
print(data.isna().sum())<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 381109 entries, 0 to 381108
Data columns (total 12 columns):
 #   Column                Non-Null Count   Dtype  
---  ------                --------------   -----  
 0   id                    381109 non-null  int64  
 1   Gender                381109 non-null  object 
 2   Age                   381109 non-null  int64  
 3   Driving_License       381109 non-null  int64  
 4   Region_Code           381109 non-null  float64
 5   Previously_Insured    381109 non-null  int64  
 6   Vehicle_Age           381109 non-null  object 
 7   Vehicle_Damage        381109 non-null  object 
 8   Annual_Premium        381109 non-null  float64
 9   Policy_Sales_Channel  381109 non-null  float64
 10  Vintage               381109 non-null  int64  
 11  Response              381109 non-null  int64  
dtypes: float64(3), int64(6), object(3)
memory usage: 34.9+ MB
None

id                      0
Gender                  0
Age                     0
Driving_Licens[...]<jupyter_text>Dataset has 381109 rows and 12 columns. There is no missing values present in dataset.<jupyter_code>#Statistical Summary of Numeric Variables
data.describe()<jupyter_output><empty_output><jupyter_text>**Data Visualizations** **Target Variable**<jupyter_code>data['Response'].value_counts()<jupyter_output><empty_output><jupyter_text>0 --> Customer is interested.1 --> Customer is not interested .<jupyter_code>sns.set_theme(style="darkgrid")
sns.countplot(data['Response'],data=data)<jupyter_output>/usr/local/lib/python3.7/dist-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning<jupyter_text>The data is highly imbalanced.<jupyter_code>plt.figure(figsize = (13,5))
plt.subplot(1,2,1)
sns.countplot(data['Gender'],palette='husl')
plt.title("count of male and female")
plt.subplot(1,2,2)
sns.countplot(data['Gender'], hue = data['Response'],palette="husl")
plt.title("Response in Male and female category")
plt.show()<jupyter_output>/usr/local/lib/python3.7/dist-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning
/usr/local/lib/python3.7/dist-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning<jupyter_text>* The gender variable in the dataset is almost equally distributed.*Male category is slightly greater than that of female and chances of buying the insurance is also little high. **Age Vs Response**<jupyter_code>#Age VS Response
plt.figure(figsize=(20,10))
sns.countplot(x='Age',hue='Response',data=data)
sns.boxplot(data['Age'])<jupyter_output>/usr/local/lib/python3.7/dist-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning<jupyter_text>* Young people below 30 are not interested in vehicle insurance. Reasons could be lack of experience, less maturity level and they don't have expensive vehicles yet.* People aged between 30-60 are more likely to be interested.* From the boxplot we can see that there no outlier in the data. **Driving License**<jupyter_code>sns.countplot(data['Driving_License'],hue=data['Response'])<jupyter_output>/usr/local/lib/python3.7/dist-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning<jupyter_text>* Customers who are interested in Vehicle Insurance almost all have driving license **Previously_Insured Vs Response**<jupyter_code>sns.countplot(x='Previously_Insured',hue='Response',data=data,palette='husl')<jupyter_output><empty_output><jupyter_text>* Customer who are not perviosly insured are likely to be inetrested **Vehicle_Age Vs Response**<jupyter_code>sns.countplot(x='Vehicle_Age',hue='Response',data=data,palette='husl')<jupyter_output><empty_output><jupyter_text>* Customers with vechicle age 1-2 years are more likely to interested as compared to the other two* Customers with with Vehicle_Age <1 years have very less chance of buying Insurance **Annual_Premium**<jupyter_code>plt.figure(figsize=(13,7))
plt.subplot(2,1,1)
sns.distplot(data['Annual_Premium'], color='Green')
plt.title("Distribution of Annual premium")
plt.show()
plt.figure(figsize=(13,7))
plt.subplot(2,1,2)
sns.boxplot(data['Annual_Premium'])
plt.title("boxplot of Annual premium")
plt.show()<jupyter_output>/usr/local/lib/python3.7/dist-packages/seaborn/distributions.py:2619: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).
  warnings.warn(msg, FutureWarning)<jupyter_text>* From the distribution plot we can infer that the annual premimum variable is right skewed* From the boxplot we can observe lot of outliers in the variable **Correlation Matrix**<jupyter_code>corr = data.corr()

f, ax = plt.subplots(figsize=(10,10))

sns.heatmap(corr, ax=ax, annot=True,linewidths=3,cmap='YlGn')

plt.title("Pearson correlation of Features", y=1.05, size=15)<jupyter_output><empty_output><jupyter_text>* Target variable is not much affected by Vintage variable. we can drop least correlated variable.  **3.Prepare Data** **Label encoding**<jupyter_code>#changing categorical value to numerical values
labelEncoder= LabelEncoder()
data['Gender'] = labelEncoder.fit_transform(data['Gender'])
data['Vehicle_Age'] = labelEncoder.fit_transform(data['Vehicle_Age'])
data['Vehicle_Damage'] = labelEncoder.fit_transform(data['Vehicle_Damage'])
data
data.dtypes<jupyter_output><empty_output><jupyter_text>**Checking duplicate rows**<jupyter_code>duplicate=data[data.duplicated()]
print(duplicate)<jupyter_output>Empty DataFrame
Columns: [id, Gender, Age, Driving_License, Region_Code, Previously_Insured, Vehicle_Age, Vehicle_Damage, Annual_Premium, Policy_Sales_Channel, Vintage, Response]
Index: []<jupyter_text>* there is no duplicated rows in the dataset **Seprating dependent and independent variables**<jupyter_code>x=data.drop(['Response'],axis=1) #contain all  independent variable
y=data['Response']           #dependent variable<jupyter_output><empty_output><jupyter_text>**Feature Selection**<jupyter_code>from sklearn.ensemble import ExtraTreesClassifier
model = ExtraTreesClassifier()
model.fit(x,y)
print(model.feature_importances_) #use inbuilt class feature_importances of tree based clas
#plot graph of feature importances for better visualization
feat_importances = pd.Series(model.feature_importances_, index=x.columns)
feat_importances.nlargest(11).plot(kind='barh')
plt.show()<jupyter_output>[0.19332928 0.00612124 0.13890651 0.00045794 0.09429041 0.05353634
 0.01715164 0.07523108 0.1637687  0.07037963 0.18682723]<jupyter_text>* We can remove less important features from the data set<jupyter_code>x=x.drop(['Driving_License','Gender'],axis=1)<jupyter_output><empty_output><jupyter_text>**Handling Imbalanced data** * When observation in one class is higher than the observation in other classes then there exists a class imbalance. We can clearly see that there is a huge difference between the data set. Solving this issue we use resampling technique.<jupyter_code>randomsample=  RandomOverSampler()
x_new,y_new=randomsample.fit_resample(x,y)

from collections import Counter
print('Original dataset shape {}'.format(Counter(y)))
print('Resampled dataset shape {}'.format(Counter(y_new)))
sns.countplot(y_new,palette='husl')<jupyter_output>Original dataset shape Counter({0: 334399, 1: 46710})
Resampled dataset shape Counter({1: 334399, 0: 334399})<jupyter_text>**Spliting of data into train and test data**<jupyter_code>#dividing the dataset into training and testing
xtrain,xtest,ytrain,ytest=train_test_split(x_new,y_new,test_size=.30,random_state=0)
print(xtrain.shape,xtest.shape,ytrain.shape,ytest.shape)

#feature scaling
from sklearn.preprocessing import StandardScaler
scaler=StandardScaler()
xtrain=scaler.fit_transform(xtrain)
xtest=scaler.transform(xtest)<jupyter_output>(468158, 9) (200640, 9) (468158,) (200640,)<jupyter_text>**4._Model Selection** * Problem can be identified as Binary Classification (wheather customer opts for vehicle insurance or not)* Dataset has more than 300k records* cannot go with SVM Classifier as it takes more time to train as dataset increase* Idea is to start selection of models as: 1.Logistic Regression 2.Random Forest 3.XGBClassifier **1.Logistic Regression**<jupyter_code>model=LogisticRegression()
model=model.fit(xtrain,ytrain)
pred=model.predict(xtest)
lr_probability =model.predict_proba(xtest)[:,1]


acc_lr=accuracy_score(ytest,pred)
recall_lr=recall_score(ytest,pred)
precision_lr=precision_score(ytest,pred)
f1score_lr=f1_score(ytest,pred)
AUC_LR=roc_auc_score(pred,ytest)
#print accuracy and Auc values of model
print('Model Name--->Logistic Regression')
print("Accuracy : ", accuracy_score(ytest,pred))
print("ROC_AUC Score:",AUC_LR)
print(f'classification_report \n{classification_report(pred,ytest)}')<jupyter_output>Model Name--->Logistic Regression
Accuracy :  0.7840161483253588
ROC_AUC Score: 0.8343713998032232
classification_report 
              precision    recall  f1-score   support

           0       0.59      0.96      0.73     61395
           1       0.98      0.70      0.82    139245

    accuracy                           0.78    200640
   macro avg       0.78      0.83      0.78    200640
weighted avg       0.86      0.78      0.79    200640<jupyter_text>**ROC Curve**<jupyter_code>from sklearn.metrics import roc_curve
fpr, tpr, _ = roc_curve(ytest, lr_probability)

plt.title('Linear Regression ROC curve')
plt.xlabel('FPR (Precision)')
plt.ylabel('TPR (Recall)')

plt.plot(fpr,tpr)
plt.plot((0,1), ls='dashed',color='black')
plt.show()

cm=confusion_matrix(ytest,pred)
print(cm)
sns.heatmap(cm,annot=True,cmap='BuPu')<jupyter_output>[[59194 41134]
 [ 2201 98111]]<jupyter_text>**2.RandomForest Classifier**<jupyter_code>from sklearn.ensemble import RandomForestClassifier

randomforest = RandomForestClassifier()
randomforest=randomforest.fit(xtrain, ytrain)
y_pred = randomforest.predict(xtest)
RF_probability = randomforest.predict_proba(xtest)[:,1]



AUC_RF=roc_auc_score(y_pred,ytest)
acc_rf=accuracy_score(ytest,y_pred)
recall_rf=recall_score(ytest,y_pred)
precision_rf=precision_score(ytest,y_pred)
f1score_rf=f1_score(ytest,y_pred)
print('Model Name--->Random Forest Classifier ')
print("Accuracy : ",accuracy_score(y_pred,ytest))
print("ROC_AUC Score:",AUC_RF)
print(f'classification_report \n{classification_report(y_pred,ytest)}')
from sklearn.metrics import roc_curve
fpr, tpr, _ = roc_curve(ytest, RF_probability)

plt.title('Linear Regression ROC curve')
plt.xlabel('FPR (Precision)')
plt.ylabel('TPR (Recall)')

plt.plot(fpr,tpr)
plt.plot((0,1), ls='dashed',color='black')
plt.show()

cm=confusion_matrix(y_pred,ytest)
print(cm)
sns.heatmap(cm,annot=True,cmap='RdPu')<jupyter_output>[[ 89511    294]
 [ 10817 100018]]<jupyter_text>**3. XGBClassifier**<jupyter_code>from xgboost import XGBClassifier
xgb=XGBClassifier()
XGB_fit=xgb.fit(xtrain, ytrain)
y_predict = XGB_fit.predict(xtest)
XGB_probability = XGB_fit.predict_proba(xtest)[:,1]

acc_xgb=accuracy_score(ytest,y_predict)
recall_xgb=recall_score(ytest,y_predict)
precision_xgb=precision_score(ytest,y_predict)
f1score_xgb=f1_score(ytest,y_predict)

AUC_xgb=roc_auc_score(y_predict,ytest)

print(f'Model Name--->XGBClassifier')
print("Accuracy : ",accuracy_score(y_predict,ytest))
print("ROC_AUC Score:",AUC_xgb)
print(f'classification_report \n{classification_report(y_predict,ytest)}')
from sklearn.metrics import roc_curve
fpr, tpr, _ = roc_curve(ytest, XGB_probability)

plt.title('XGBoost ROC curve')
plt.xlabel('FPR (Precision)')
plt.ylabel('TPR (Recall)')

plt.plot(fpr,tpr)
plt.plot((0,1), ls='dashed',color='black')
plt.show()

#it helps to identify how many values are classified correctly
cm=confusion_matrix(ytest,y_predict)
print(cm)
sns.heatmap(cm,annot=True,cmap='GnBu')<jupyter_output>[[66459 33869]
 [ 6577 93735]]<jupyter_text>**Comparing the model**<jupyter_code>ind=['Logistic regression','Randomforest','XGBClassifier']
data={"Accuracy":[acc_lr,acc_rf,acc_xgb],"Recall":[recall_lr,recall_rf,recall_xgb],"Precision":[precision_lr,precision_rf,precision_xgb],
    'f1_score':[f1score_lr,f1score_rf,f1score_xgb],"ROC_AUC":[AUC_LR,AUC_RF,AUC_xgb]}
result=pd.DataFrame(data=data,index=ind)
result<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09875206564476609, 'fraction_numerical': 0.036127414667502425, 'mean_word_length': 5.8931657501963866, 'pattern_counts': {'":': 4, '<': 89, '<?xml version=': 0, '>': 94, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '25', 'dir': 'jupyter-structured-clean-dedup', 'id': '582556', 'license': 'BSD-Source-Code', 'n_tokens_mistral': 6046, 'n_tokens_neox': 5460, 'n_words': 1613, 'path': 'HEALTH_INSURANCE_CROSS_SELL_PREDICTION.ipynb', 'repo_name': 'Dayanand07/Health-Insurance-Cross-Sell-Prediction', 'size': '17549'}