<jupyter_start><jupyter_text><NAME>’s Safe Driver PredictionThe aim of this compitation is to predict probability that a driver will intiate an auto insurance claim next year.A more accurate prediction will allow them to further tailor their prices, and hopefully make auto insurance coverage more accessible to more drivers. **Steps**1. [Read data set](Read-data-set)2. [Explore data set](Explore-data-set)3. [Correlation plot](Correlation-plot)4. [Missing value is data set](Missing-value-is-data-set)5. [Convert variables into category type](Convert-variables-into-category-type)6. [Univariate analysis](Univariate-analysis)7. [Descrictive Statistic Features](Descrictive-Statistic-Features)8. [Determine outliers in dataset](Determine-outliers-in-dataset)9. [One Hot Encoding](One-Hot-Encoding)10. [Split data set](Split-data-set)11. [Hyperparameter tuning](Hyperparameter-tuning)12. [Logistic Regression model](Logistic-Regression-model)13. [Model performance](Model-performance)14. [Reciever Operating Charactaristics](Reciever-Operating-Charactaristics)15. [Predict for unseen data set](Predict-for-unseen-data-set)  Import library<jupyter_code>#Import library
import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix, roc_auc_score ,roc_curve,auc
from sklearn.model_selection import StratifiedKFold,GridSearchCV
import missingno as mssno
seed =45
% matplotlib inline<jupyter_output><empty_output><jupyter_text>Read data set<jupyter_code>path = '../input/'
#path = 'dataset/'
train = pd.read_csv(path+'train.csv',na_values=-1)
test = pd.read_csv(path+'test.csv',na_values=-1)
print('Number rows and columns:',train.shape)
print('Number rows and columns:',test.shape)<jupyter_output><empty_output><jupyter_text>Porto Seguro provided close to 600k and 900k observation of train and test dataset respectively. They were 57 feature anonymized in order to protect company trade secrets, but they were given bit informaation about  The train and test data set contains feature with similar grouping are tagged with (e.g., ind, reg, car, cat, calc, bin). Values of  -1 indicate that the feature was missing from the observation. Explore data set<jupyter_code>train.head(3).T<jupyter_output><empty_output><jupyter_text>Target varaiable<jupyter_code>plt.figure(figsize=(10,3))
sns.countplot(train['target'],palette='rainbow')
plt.xlabel('Target')

train['target'].value_counts()<jupyter_output><empty_output><jupyter_text>The 'target' variable in imbalanced. The target column in data set is whether or not claim was filed for that policy holder. The target variable is quite unbalanced, with only  %4 of  policyholders in training data filing claim within the year.  Correlation plotCorrelation is a measure bivariate analysis that measure the strength of assciation between variable and direction of relationship.In terms of strength of relationship, the value of the correlation coefficient varies between +1 and -1<jupyter_code>cor = train.corr()
plt.figure(figsize=(16,10))
sns.heatmap(cor,cmap='Set1')<jupyter_output><empty_output><jupyter_text>> The correlation coefficient for **ps_calc** is 0,so we will drop these from our dataset.<jupyter_code>ps_cal = train.columns[train.columns.str.startswith('ps_calc')] 
train = train.drop(ps_cal,axis =1)
test = test.drop(ps_cal,axis=1)
train.shape<jupyter_output><empty_output><jupyter_text>Missing value is data set>Values of -1 indicate that the feature was missing from the observation. The target columns signifies whether or not a claim was filed for that policy holder.<jupyter_code>k= pd.DataFrame()
k['train']= train.isnull().sum()
k['test'] = test.isnull().sum()
fig,ax = plt.subplots(figsize=(16,5))
k.plot(kind='bar',ax=ax)<jupyter_output><empty_output><jupyter_text>Missing value in test train data set are in same propotion and same column<jupyter_code>mssno.bar(train,color='y',figsize=(16,4),fontsize=12)
mssno.bar(test,color='b',figsize=(16,4),fontsize=12)
mssno.matrix(train)<jupyter_output><empty_output><jupyter_text>Replace missing value with mode<jupyter_code>def missing_value(df):
    col = df.columns
    for i in col:
        if df[i].isnull().sum()>0:
            df[i].fillna(df[i].mode()[0],inplace=True)
missing_value(train)
missing_value(test)<jupyter_output><empty_output><jupyter_text>Convert variables into category type<jupyter_code>def basic_details(df):
    b = pd.DataFrame()
    b['Missing value'] = df.isnull().sum()
    b['N unique value'] = df.nunique()
    b['dtype'] = df.dtypes
    return b
basic_details(train)<jupyter_output><empty_output><jupyter_text>>The unique value of "ps_car_11_cat" is maximum in the data set is 104<jupyter_code>def category_type(df):
    col = df.columns
    for i in col:
        if df[i].nunique()<=104:
            df[i] = df[i].astype('category')
category_type(train)
category_type(test)<jupyter_output><empty_output><jupyter_text>Univariate analysis<jupyter_code>cat_col = [col for col in train.columns if '_cat' in col]
print(cat_col)
fig ,ax = plt.subplots(1,2,figsize=(14,2))
ax1,ax2, = ax.flatten()
sns.countplot(train['ps_ind_02_cat'],palette='rainbow',ax=ax1)
sns.countplot(train['ps_ind_04_cat'],palette='summer',ax=ax2)
fig,ax = plt.subplots(figsize=(14,2))
sns.countplot(train['ps_ind_05_cat'],palette='rainbow',ax=ax)
fig,ax = plt.subplots(2,2,figsize=(14,4))
ax1,ax2,ax3,ax4 = ax.flatten()
sns.countplot(train['ps_car_01_cat'],palette='rainbow',ax=ax1)
sns.countplot(train['ps_car_02_cat'],palette='summer',ax=ax2)
sns.countplot(train['ps_car_03_cat'],palette='summer',ax=ax3)
sns.countplot(train['ps_car_04_cat'],palette='rainbow',ax=ax4)
fig,ax = plt.subplots(2,2,figsize = (14,4))
ax1,ax2,ax3,ax4 = ax.flatten()
sns.countplot(train['ps_car_05_cat'],palette='summer',ax=ax1)
sns.countplot(train['ps_car_06_cat'],palette='rainbow',ax=ax2)
sns.countplot(train['ps_car_07_cat'],palette='summer',ax=ax3)
sns.countplot(train['ps_car_08_cat'],palette='rainbow',ax=ax4)

fig, ax = plt.subplots(1,2,figsize=(14,2))
ax1,ax2 = ax.flatten()
sns.countplot(train['ps_car_09_cat'],palette='rainbow',ax=ax1)
sns.countplot(train['ps_car_10_cat'],palette='gist_rainbow',ax=ax2)
fig,ax = plt.subplots(figsize=(14,3))
sns.countplot(train['ps_car_11_cat'],palette='rainbow',ax=ax)
bin_col = [col for col in train.columns if 'bin' in col]
print(bin_col)
fig,ax = plt.subplots(3,3,figsize=(15,14),sharex='all')
ax1,ax2,ax3,ax4,ax5,ax6,ax7,ax8,ax9 = ax.flatten()
sns.countplot(train['ps_ind_06_bin'],palette='rainbow',ax=ax1)
sns.countplot(train['ps_ind_07_bin'],palette='summer',ax=ax2)
sns.countplot(train['ps_ind_08_bin'],palette='gist_rainbow',ax=ax3)
sns.countplot(train['ps_ind_09_bin'],palette='summer',ax=ax4)
sns.countplot(train['ps_ind_10_bin'],palette='rainbow',ax=ax5)
sns.countplot(train['ps_ind_11_bin'],palette='gist_rainbow',ax=ax6)
sns.countplot(train['ps_ind_12_bin'],palette='coolwarm',ax=ax7)
sns.countplot(train['ps_ind_13_bin'],palette='gist_rainbow',ax=ax8)
sns.countplot(train['ps_ind_16_bin'],palette='rainbow',ax=ax9)
fig,ax = plt.subplots(1,2,figsize=(14,6))
ax1,ax2 = ax.flatten()
sns.countplot(train['ps_ind_17_bin'],palette='coolwarm',ax=ax1)
sns.countplot(train['ps_ind_18_bin'],palette='gist_rainbow',ax=ax2)
tot_cat_col = list(train.select_dtypes(include=['category']).columns)

other_cat_col = [c for c in tot_cat_col if c not in cat_col+ bin_col]
other_cat_col
fig,ax = plt.subplots(2,2,figsize=(14,6))
ax1,ax2,ax3,ax4 = ax.flatten()
sns.countplot(data=train,x='ps_ind_01',palette='rainbow',ax=ax1)
sns.countplot(data=train,x='ps_ind_03',palette='gist_rainbow',ax=ax2)
sns.countplot(data=train,x='ps_ind_14',palette='gist_rainbow',ax=ax3)
sns.countplot(data=train,x='ps_ind_15',palette='rainbow',ax=ax4)
fig,ax = plt.subplots(2,2,figsize=(14,6))
ax1,ax2,ax3,ax4 =ax.flatten()
sns.countplot(data=train,x='ps_reg_01',palette='gist_rainbow',ax=ax1)
sns.countplot(data=train,x='ps_reg_02',palette='rainbow',ax=ax2)
sns.countplot(data=train,x='ps_car_11',palette='summer',ax=ax3)
sns.countplot(data=train,x='ps_car_15',palette='gist_rainbow',ax=ax4)
plt.xticks(rotation=90)
num_col = [c for c in train.columns if c not in tot_cat_col]
num_col.remove('id')
num_col
train['ps_reg_03'].describe()
fig,ax = plt.subplots(2,2,figsize=(14,8))
ax1,ax2,ax3,ax4 = ax.flatten()
sns.distplot(train['ps_reg_03'],bins=100,color='red',ax=ax1)
sns.boxplot(x ='ps_reg_03',y='target',data=train,ax=ax2)
sns.violinplot(x ='ps_reg_03',y='target',data=train,ax=ax3)
sns.pointplot(x= 'ps_reg_03',y='target',data=train,ax=ax4)<jupyter_output><empty_output><jupyter_text>ps_reg_03 has outlier data points<jupyter_code>train['ps_car_12'].describe()
fig,ax = plt.subplots(2,2,figsize=(14,8))
ax1,ax2,ax3,ax4 = ax.flatten()
sns.distplot(train['ps_car_12'],bins=50,ax=ax1)
sns.boxplot(x='ps_car_12',y='target',data=train,ax=ax2)
sns.violinplot(x='ps_car_12',y='target',data=train,ax=ax3)
sns.pointplot(x='ps_car_12',y='target',data=train,ax=ax4)
train['ps_car_13'].describe()
fig,ax = plt.subplots(2,2,figsize=(14,8))
ax1,ax2,ax3,ax4 = ax.flatten()
sns.distplot(train['ps_car_13'],bins=120,ax=ax1)
sns.boxplot(x='ps_car_13',y='target',data=train,ax=ax2)
sns.violinplot(x='ps_car_13',y='target',data=train,ax=ax3)
sns.pointplot(x='ps_car_13',y='target',data=train,ax=ax4)
train['ps_car_14'].describe()
fig,ax = plt.subplots(2,2,figsize=(14,8))
ax1,ax2,ax3,ax4 = ax.flatten()
sns.distplot(train['ps_car_14'],bins=120,ax=ax1)
sns.boxplot(x='ps_car_14',y='target',data=train,ax=ax2)
sns.violinplot(x='ps_car_14',y='target',data=train,ax=ax3)
sns.pointplot(x='ps_car_14',y='target',data=train,ax=ax4)<jupyter_output><empty_output><jupyter_text>Descrictive Statistic Features<jupyter_code>def descrictive_stat_feat(df):
    df = pd.DataFrame(df)
    dcol= [c for c in train.columns if train[c].nunique()>=10]
    dcol.remove('id')   
    d_median = df[dcol].median(axis=0)
    d_mean = df[dcol].mean(axis=0)
    q1 = df[dcol].apply(np.float32).quantile(0.25)
    q3 = df[dcol].apply(np.float32).quantile(0.75)
    
    #Add mean and median column to data set having more then 10 categories
    for c in dcol:
        df[c+str('_median_range')] = (df[c].astype(np.float32).values > d_median[c]).astype(np.int8)
        df[c+str('_mean_range')] = (df[c].astype(np.float32).values > d_mean[c]).astype(np.int8)
        df[c+str('_q1')] = (df[c].astype(np.float32).values < q1[c]).astype(np.int8)
        df[c+str('_q3')] = (df[c].astype(np.float32).values > q3[c]).astype(np.int8)
    return df
train = descrictive_stat_feat(train)
test = descrictive_stat_feat(test)<jupyter_output><empty_output><jupyter_text>Correlation plot<jupyter_code>cor = train[num_col].corr()
plt.figure(figsize=(10,4))
sns.heatmap(cor,annot=True)
plt.tight_layout()<jupyter_output><empty_output><jupyter_text>Determine outliers in datasetThe extreme observations in data set which resembles completely different behavoir from the rest of data point are called outliers. The outliers present in numeric feature are replaced by 1%/99% of feature.<jupyter_code>def outlier(df,columns):
    for i in columns:
        quartile_1,quartile_3 = np.percentile(df[i],[25,75])
        quartile_f,quartile_l = np.percentile(df[i],[1,99])
        IQR = quartile_3-quartile_1
        lower_bound = quartile_1 - (1.5*IQR)
        upper_bound = quartile_3 + (1.5*IQR)
        print(i,lower_bound,upper_bound,quartile_f,quartile_l)
                
        df[i].loc[df[i] < lower_bound] = quartile_f
        df[i].loc[df[i] > upper_bound] = quartile_l
        
outlier(train,num_col)
outlier(test,num_col)<jupyter_output><empty_output><jupyter_text>One Hot EncodingA One hot encoding is a representation of categorical variable as binary vectors.It allows the representation of categorical data to be more expresive. This first requires that the categorical values be mapped to integer values, that is label encoding. Then, each integer value is represented as a binary vector that is all zero values except the index of the integer, which is marked with a 1.The Dummy variable trap is a scenario in which the independent variable are multicollinear, a scenario in which two or more variables are highly correlated in simple term one variable can be predicted from the others.<jupyter_code>def OHE(df1,df2,column):
    cat_col = column
    #cat_col = df.select_dtypes(include =['category']).columns
    len_df1 = df1.shape[0]
    
    df = pd.concat([df1,df2],ignore_index=True)
    c2,c3 = [],{}
    
    print('Categorical feature',len(column))
    for c in cat_col:
        if df[c].nunique()>2 :
            c2.append(c)
            c3[c] = 'ohe_'+c
    
    df = pd.get_dummies(df, prefix=c3, columns=c2,drop_first=True)

    df1 = df.loc[:len_df1-1]
    df2 = df.loc[len_df1:]
    print('Train',df1.shape)
    print('Test',df2.shape)
    return df1,df2
train1,test1 = OHE(train,test,tot_cat_col)<jupyter_output><empty_output><jupyter_text>Split data set<jupyter_code>X = train1.drop(['target','id'],axis=1)
y = train1['target'].astype('category')
x_test = test1.drop(['target','id'],axis=1)
del train1,test1<jupyter_output><empty_output><jupyter_text>Hyperparameter tuning<jupyter_code>#Grid search
"""logreg = LogisticRegression(class_weight='balanced')
param = {'C':[0.001,0.003,0.005,0.01,0.03,0.05,0.1,0.3,0.5,1]}
clf = GridSearchCV(logreg,param,scoring='roc_auc',refit=True,cv=3)
clf.fit(X,y)
print('Best roc_auc: {:.4}, with best C: {}'.format(clf.best_score_, clf.best_params_['C'])) """<jupyter_output><empty_output><jupyter_text>Logistic Regression modelLogistic is suprevised algorithom used for discret type target variable. Unlike linear regression whose target varible is continues real number, logsitic regression transform transform target variable using sigmoid function to return probability value. If value predicted by the function is greater than 50% than it belong to class 1 or else 0.Logistic regression is used for modelling. The data set is split using Stratified Kfold. In each split model is created and predicted using that model. The final predicted value is average of all model.<jupyter_code>kf = StratifiedKFold(n_splits=5,random_state=seed,shuffle=True)
pred_test_full=0
cv_score=[]
i=1
for train_index,test_index in kf.split(X,y):    
    print('\n{} of kfold {}'.format(i,kf.n_splits))
    xtr,xvl = X.loc[train_index],X.loc[test_index]
    ytr,yvl = y[train_index],y[test_index]
    
    lr = LogisticRegression(class_weight='balanced',C=0.003)
    lr.fit(xtr, ytr)
    pred_test = lr.predict_proba(xvl)[:,1]
    score = roc_auc_score(yvl,pred_test)
    print('roc_auc_score',score)
    cv_score.append(score)
    pred_test_full += lr.predict_proba(x_test)[:,1]
    i+=1<jupyter_output><empty_output><jupyter_text>Model performance<jupyter_code>print('Confusion matrix\n',confusion_matrix(yvl,lr.predict(xvl)))
print('Cv',cv_score,'\nMean cv Score',np.mean(cv_score))<jupyter_output><empty_output><jupyter_text>Reciever Operating Charactaristics<jupyter_code>proba = lr.predict_proba(xvl)[:,1]
fpr,tpr, threshold = roc_curve(yvl,proba)
auc_val = auc(fpr,tpr)

plt.figure(figsize=(14,8))
plt.title('Reciever Operating Charactaristics')
plt.plot(fpr,tpr,'b',label = 'AUC = %0.2f' % auc_val)
plt.legend(loc='lower right')
plt.plot([0,1],[0,1],'r--')
plt.ylabel('True positive rate')
plt.xlabel('False positive rate')<jupyter_output><empty_output><jupyter_text>Predict for unseen data set<jupyter_code>y_pred = pred_test_full/5
submit = pd.DataFrame({'id':test['id'],'target':y_pred})
#submit.to_csv('lr_porto.csv.gz',index=False,compression='gzip') 
submit.to_csv('lr_porto.csv',index=False) 
submit.head()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14958219047011545, 'fraction_numerical': 0.03189385724309498, 'mean_word_length': 7.247238295633877, 'pattern_counts': {'":': 0, '<': 97, '<?xml version=': 0, '>': 104, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '278743', 'license': 'MIT', 'n_tokens_mistral': 6237, 'n_tokens_neox': 5926, 'n_words': 1044, 'path': '10 poer sugero safe driver prediction/simple-logistic-model-porto.ipynb', 'repo_name': 'MLVPRASAD/KaggleProjects', 'size': '15683'}