<jupyter_start><jupyter_text>Titanic survivor predictionHi, in this notebook we are going to explore, analyze and predict whether a passenger on the Titanic survived or not. First, we should start by downloading the data.![alttext](https://media.giphy.com/media/4d8uCMJE5b8ru/giphy.gif) Download the dataYou can find the dataset [here](https://www.kaggle.com/c/titanic/data).Now let's import the first necessary libraries. Importing the libaries<jupyter_code>import math,time,datetime

import pandas as pd
import numpy as np
import missingno
import matplotlib.pyplot as plt
import seaborn as sns
from collections import Counter
from sklearn.preprocessing import OneHotEncoder, LabelEncoder, StandardScaler
import string
# Classifier
from sklearn.linear_model import LogisticRegression,LinearRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.svm import LinearSVC
from sklearn.tree import DecisionTreeClassifier
from sklearn import model_selection,preprocessing,metrics
from sklearn.model_selection import GridSearchCV
from sklearn.ensemble import RandomForestClassifier,GradientBoostingClassifier
import xgboost as xgb
# Warnings
import warnings
warnings.filterwarnings('ignore')<jupyter_output><empty_output><jupyter_text>Importing the dataNow let's import the dataset via pandas<jupyter_code>train = pd.read_csv('data/train.csv')
test = pd.read_csv('data/test.csv')

train.head()
test.tail()<jupyter_output><empty_output><jupyter_text>Data desrciption**Survival**: 0 = No, 1 = Yes**pclass (Ticket class)**: 1 = 1st, 2 = 2nd, 3 = 3rd**sex**: Sex**Age**: Age in years**sibsp**: number of siblings/spouses aboard the Titanic**parch**: number of parents/children aboard the Titanic**ticket**: Ticket number**fare**: Passenger fare**cabin**: Cabin number**embarked**: Port of Embarkation, C = Cherbourg, Q = Queenstown, S = SouthamptonColumns as described as shown above.<jupyter_code>train.describe()<jupyter_output><empty_output><jupyter_text>Outlier Detection and RemovalTo improve the classification accuracy we are now going to find Outliers in the dataset and remove them.<jupyter_code>def detect_outliers(df,n,features):
    """
    Takes a dataframe df of features and returns a list of the indices
    corresponding to the observations containing more than n outliers according
    to the Tukey method.
    """
    outlier_indices = []
    
    # iterate over features(columns)
    for col in features:
        # 1st quartile (25%)
        Q1 = np.percentile(df[col], 25)
        # 3rd quartile (75%)
        Q3 = np.percentile(df[col],75)
        # Interquartile range (IQR)
        IQR = Q3 - Q1
        
        # outlier step
        outlier_step = 1.5 * IQR
        
        # Determine a list of indices of outliers for feature col
        outlier_list_col = df[(df[col] < Q1 - outlier_step) | (df[col] > Q3 + outlier_step )].index
        
        # append the found outlier indices for col to the list of outlier indices 
        outlier_indices.extend(outlier_list_col)
        
    # select observations containing more than 2 outliers
    outlier_indices = Counter(outlier_indices)        
    multiple_outliers = list( k for k, v in outlier_indices.items() if v > n )
    
    return multiple_outliers
outliers_to_drop = detect_outliers(train,2,["Age","SibSp","Parch","Fare"])
train = train.drop(outliers_to_drop, axis = 0).reset_index(drop=True)<jupyter_output><empty_output><jupyter_text>Missing valuesNow let's find out how many missing values we have and in which feature they are.<jupyter_code>missingno.matrix(train,figsize=(30,10))<jupyter_output><empty_output><jupyter_text>From this chart we can see that **Age** has(I would argue) about 15% missing values. **Cabin** has a lot more values missing let's say about 80%. **Embarked** has just a few missing, I would argue under 1%.Now let's see how many missing values are there exactly.<jupyter_code>df_length = len(train)
print("Train total length: " +  str(df_length))
for i in train.columns:
    df_nans = df_length - train[i].value_counts().sum()
    print(i + ": " + str(df_nans) + "   " + str(round((df_nans/df_length)*100)) + "% missing")
df_length = len(test)
print("Train total length: " +  str(df_length))
for i in test.columns:
    df_nans = df_length - test[i].value_counts().sum()
    print(i + ": " + str(df_nans) + "   " + str(round((df_nans/df_length)*100)) + "% missing")<jupyter_output>Train total length: 418
PassengerId: 0   0.0% missing
Pclass: 0   0.0% missing
Name: 0   0.0% missing
Sex: 0   0.0% missing
Age: 86   21.0% missing
SibSp: 0   0.0% missing
Parch: 0   0.0% missing
Ticket: 0   0.0% missing
Fare: 1   0.0% missing
Cabin: 327   78.0% missing
Embarked: 0   0.0% missing<jupyter_text>Handling the missing data AgeNow let's focus on how to solve the problem with the missing data in the feature **age**. As I showed earlier we have 177 times missing ages in the dataset. Which is almost a quarter! So we need to find a way to replace them. After brainstorming about this problem I came up with the following plan:- 1. If the persons' sex is male and he has the word **Master** in his name we set the age to the average value of males under 21.- 2. If the persons' sex is male and he has the word **Mr** in his name and travels alone his age is set to the average age of males with **Mr** in his name who travel alone. Else: The age is set to the average age of males with the word **Mr** in his name.- 3. If the persons' sex is female and has **Miss** in her name her age is set to the average age of females under 21.- 4. If the persons' sex is female and has **Mrs** in her name and travels alone her age is set to the average age of females who travel alone and have **Mrs** in her name. Else: The age is set to the average age of females with the word **Mrs** in her name.Alright now let's develop a method which replaces the NaN's with the age set by the rules before.But first we need to fill the missing values(currently NaN) with 0's to replace them.<jupyter_code>train.Age.fillna(0,inplace=True)
test.Age.fillna(0,inplace=True)
train.head()
sum_male_under_21 = 0
sum_male_under_21_c = 0
sum_male_alone = 0
sum_male_alone_c = 0
sum_male_over_21 = 0
sum_male_over_21_c = 0

sum_female_under_21 = 0
sum_female_under_21_c = 0
sum_female_alone = 0
sum_female_alone_c = 0
sum_female_over_21 = 0
sum_female_over_21_c = 0


for i, row in train.iterrows():
    if train['Age'][i] != 0: 
        #male
        if train['Age'][i] < 21: 
            if train['Sex'][i] == 'male':
                sum_male_under_21 = sum_male_under_21 + train['Age'][i]
                sum_male_under_21_c = sum_male_under_21_c + 1
        if train['Age'][i] >= 21: 
            if train['Sex'][i] == 'male':
                sum_male_over_21 = sum_male_over_21 + train['Age'][i]
                sum_male_over_21_c = sum_male_over_21_c + 1    
        if train['Age'][i] >= 21: 
            if train['Sex'][i] == 'male':
                if train['SibSp'][i] == 0:
                    if train['Parch'][i] == 0:
                        sum_male_over_21 = sum_male_alone + train['Age'][i]
                        sum_male_over_21_c = sum_male_alone_c + 1
        #female
        if train['Age'][i] < 21: 
            if train['Sex'][i] == 'female':
                sum_female_under_21 = sum_female_under_21 + train['Age'][i]
                sum_female_under_21_c = sum_female_under_21_c + 1
        if train['Age'][i] >= 21: 
            if train['Sex'][i] == 'female':
                sum_female_over_21 = sum_female_over_21 + train['Age'][i]
                sum_female_over_21_c = sum_female_over_21_c + 1    
        if train['Age'][i] >= 21: 
            if train['Sex'][i] == 'female':
                if train['SibSp'][i] == 0:
                    if train['Parch'][i] == 0:
                        sum_female_over_21 = sum_female_alone + train['Age'][i]
                        sum_female_over_21_c = sum_female_alone_c + 1
                        
                        
avg_male_under_21 = sum_male_under_21 / sum_male_under_21_c
avg_male_over_21 = sum_male_over_21 / sum_male_over_21_c
avg_male_alone = 0
avg_female_under_21 = sum_female_under_21 / sum_female_under_21_c
avg_female_over_21 = sum_female_over_21 / sum_female_over_21_c
avg_female_alone = 0

print(avg_male_under_21,avg_male_over_21,avg_male_alone,avg_female_under_21,avg_female_over_21,avg_female_alone,sep='\n')<jupyter_output>13.246764705882354
32.0
0
11.883116883116884
30.5
0<jupyter_text>Now let's fill in the matching **ages**:<jupyter_code>c = 0
while c < 2:
    if (c == 0):
        f = train
    if (c == 1):
        f = test
    for i, row in f.iterrows():
        if f['Age'][i] == 0: 
            #male
            if f['Age'][i] < 21: 
                if f['Sex'][i] == 'male':
                    f.loc[i,'Age'] = avg_male_under_21
            if f['Age'][i] >= 21: 
                if f['Sex'][i] == 'male':
                    f.loc[i,'Age'] = avg_male_over_21
            if f['Age'][i] >= 21: 
                if f['Sex'][i] == 'male':
                    if f['SibSp'][i] == 0:
                        if f['Parch'][i] == 0:
                            f.loc[i,'Age'] = avg_male_alone
            #female
            if f['Age'][i] < 21: 
                if f['Sex'][i] == 'female':
                    f.loc[i,'Age'] = avg_female_under_21
            if f['Age'][i] >= 21: 
                if f['Sex'][i] == 'female':
                    f.loc[i,'Age'] = avg_female_over_21 
            if f['Age'][i] >= 21: 
                if f['Sex'][i] == 'female':
                    if f['SibSp'][i] == 0:
                        if f['Parch'][i] == 0:
                            f.loc[i,'Age'] = avg_female_alone
    c = c + 1                        
train.tail()
test.tail()<jupyter_output><empty_output><jupyter_text>EmbarkedFor the **Embarked** feature we know that there are only **2** rows where it is missing, so let's just drop them to fix that.<jupyter_code>train.drop(index=[61,829],inplace=True)
train.loc[train['Embarked'] == 0]<jupyter_output><empty_output><jupyter_text>Analyse dataBefore we start analyzing each feature on its own, we should start by finding out the data type of each feature:<jupyter_code>train.dtypes<jupyter_output><empty_output><jupyter_text>We see that there are just a few features who aren't in the data type of a numerical value e.g. int or float. This is going to be a good thing for the feature encoding later on(Same goes for the test dataframe).Now let's start analyzing the data, first let's start with Survived0 = passenger not survived1 = passenger survivedThis is the variable we want our machine learning model to predict based off all the others.<jupyter_code>fig = plt.figure(figsize=(20,2))
sns.countplot(y='Survived',data=train)
len(train.loc[train['Survived'] == 1])
len(train.loc[train['Survived'] == 0])<jupyter_output><empty_output><jupyter_text>Therefore we can see that more than 500 people died on the Titanic and about 350 people survived the sinking.So the probability that a randomly picked passenger survived the sinking is 340/889 so about **38%**. PclassDescription: The ticket class of the passenger.Key: 1=1st 2=2nd 3=3rdWe will look at the distribution of each feature first if we can to understand what kind of spread there is across the dataset.For example, if there are values which are completely outside of the distribution, we may not want to include them in our model. So let's check that out.<jupyter_code>sns.distplot(train.Pclass)<jupyter_output><empty_output><jupyter_text>We see that this is not really the case so we'll use this feature for our prediction later on. SexLet's compare the distribution of the genders.<jupyter_code>fig = plt.figure(figsize=(20,3))
sns.countplot(y='Sex',data=train)<jupyter_output><empty_output><jupyter_text>Now let's check out how many of the male's and female's have survived.<jupyter_code>g = sns.catplot(x='Sex',y='Survived',data=train,height=6,kind='bar') 
g.set_ylabels('Survival probability')<jupyter_output><empty_output><jupyter_text>So as we see the probability that a randomly picked passenger survived the Titanic is 4-times higher if the passenger is female. Age Now let's check how the different ages are spread in the dataset.<jupyter_code>plt.figure(figsize=(10,5))
# plots a kernel density estimate of the subset of the 1st class passangers's age
train.Age.plot(kind='kde')    
train['Age'].sum() / len(train)<jupyter_output><empty_output><jupyter_text>Therefore we know that the average age aboard the Titanic is about **26 years**.Now let's find out how the probability to survive the sinking is with view on the different ages.<jupyter_code>fig = plt.figure(figsize=(20,10))
sns.set(style='darkgrid')
sns.lineplot(x='Age',y='Survived',data=train,err_style="bars")<jupyter_output><empty_output><jupyter_text>Out of this we can see that if a passenger is between 65 and 74 he's going to die during the sinking, this is the worst age of all! For the rest take a look at the chart yourself :) SibSpNow let's check how many sibilings/spouses are aboard with the passenger.<jupyter_code>fig = plt.figure(figsize=(20,10))
g = sns.countplot(x='SibSp',data=train)<jupyter_output><empty_output><jupyter_text>Therefore we can see that most passengers traveled alone. ParchNow let's do the same as we did with the sibilings/spouses with parents/children aboard the Titanic.<jupyter_code>fig = plt.figure(figsize=(20,10))
g = sns.countplot(x='Parch',data=train)<jupyter_output><empty_output><jupyter_text>TicketAs you've might observed earlier there are a **lot** of different ticket types:<jupyter_code>train.Ticket.value_counts()<jupyter_output><empty_output><jupyter_text>So there are **680** unique ticket types. Wow! That is a lot, and really not easy to simplify in the feature encoding, later on, so we won't use this feature in it's natural form in the prediction later on. FareDescription: How much the ticket cost for every individual passenger.<jupyter_code>fig = plt.figure(figsize=(20,45))
sns.countplot(y='Fare',data=train)<jupyter_output><empty_output><jupyter_text>Fun fact: This is the average ticket price for a fare on the Titanic.<jupyter_code>train.Fare.sum() / len(train)<jupyter_output><empty_output><jupyter_text>EmbarkedDescription: Port's were the passengers had go on board of the Titanic.Possible port's are: C = Cherbourg, Q = Queenstown, S = Southampton<jupyter_code>fig = plt.figure(figsize=(10,8))
sns.countplot(x='Embarked',data=train)<jupyter_output><empty_output><jupyter_text>Feature EngineeringI'm going to engineer a couple of new features to improve the accuracy. For all catigorical features I'm going to add dummy data via pandas to create new columns(again for better accuracy).<jupyter_code>df_all = train.append(test, ignore_index=True)
df_all.head()<jupyter_output><empty_output><jupyter_text>Age and Fare in test data<jupyter_code>df_all[ 'Age' ] = df_all.Age.fillna( df_all.Age.mean() )

# Fill missing values of Fare with the average of Fare (mean)
df_all[ 'Fare' ] = df_all.Fare.fillna( df_all.Fare.mean() )

df_all.head()<jupyter_output><empty_output><jupyter_text>Family Size<jupyter_code>family = pd.DataFrame()

family['FamilySize'] = df_all['SibSp'] + df_all.Parch + 1

family[ 'Family_Single' ] = family[ 'FamilySize' ].map( lambda s : 1 if s == 1 else 0 )
family[ 'Family_Small' ]  = family[ 'FamilySize' ].map( lambda s : 1 if 2 <= s <= 4 else 0 )
family[ 'Family_Large' ]  = family[ 'FamilySize' ].map( lambda s : 1 if 5 <= s else 0 )

family.head()<jupyter_output><empty_output><jupyter_text>Title<jupyter_code>title = pd.DataFrame()
# we extract the title from each name
title[ 'Title' ] = df_all[ 'Name' ].map( lambda name: name.split( ',' )[1].split( '.' )[0].strip() )

# a map of more aggregated titles
Title_Dictionary = {
                    "Capt":       "Officer",
                    "Col":        "Officer",
                    "Major":      "Officer",
                    "Jonkheer":   "Royalty",
                    "Don":        "Royalty",
                    "Sir" :       "Royalty",
                    "Dr":         "Officer",
                    "Rev":        "Officer",
                    "the Countess":"Royalty",
                    "Dona":       "Royalty",
                    "Mme":        "Mrs",
                    "Mlle":       "Miss",
                    "Ms":         "Mrs",
                    "Mr" :        "Mr",
                    "Mrs" :       "Mrs",
                    "Miss" :      "Miss",
                    "Master" :    "Master",
                    "Lady" :      "Royalty"

                    }

# we map each title
title[ 'Title' ] = title.Title.map( Title_Dictionary )
title = pd.get_dummies( title.Title )

title.head()<jupyter_output><empty_output><jupyter_text>Cabin<jupyter_code>cabin = pd.DataFrame()

# replacing missing cabins with U (for Uknown)
cabin[ 'Cabin' ] = df_all.Cabin.fillna( 'U' )

# mapping each Cabin value with the cabin letter
cabin[ 'Cabin' ] = cabin[ 'Cabin' ].map( lambda c : c[0] )

# dummy encoding ...
cabin = pd.get_dummies( cabin['Cabin'] , prefix = 'Cabin' )

cabin.head()<jupyter_output><empty_output><jupyter_text>Ticket<jupyter_code>def cleanTicket( ticket ):
    ticket = ticket.replace( '.' , '' )
    ticket = ticket.replace( '/' , '' )
    ticket = ticket.split()
    ticket = map( lambda t : t.strip() , ticket )
    ticket = list(filter( lambda t : not t.isdigit() , ticket ))
    if len( ticket ) > 0:
        return ticket[0]
    else: 
        return 'XXX'

ticket = pd.DataFrame()

# Extracting dummy variables from tickets:
ticket[ 'Ticket' ] = df_all[ 'Ticket' ].map( cleanTicket )
ticket = pd.get_dummies( ticket[ 'Ticket' ] , prefix = 'Ticket' )

ticket.shape
ticket.head()<jupyter_output><empty_output><jupyter_text>Embarked<jupyter_code>embarked = pd.get_dummies( df_all.Embarked , prefix='Embarked' )
embarked.head()<jupyter_output><empty_output><jupyter_text>Pclass<jupyter_code>pclass = pd.get_dummies( df_all.Pclass , prefix='Pclass' )
pclass.head()<jupyter_output><empty_output><jupyter_text>Sex<jupyter_code>sex = pd.get_dummies(df_all.Sex,prefix='Sex')
sex.head()<jupyter_output><empty_output><jupyter_text>IsAlone<jupyter_code>is_alone_l = []
for i,row in df_all.iterrows():
  if row.Parch == 0 and row.SibSp == 0:
    is_alone_l.append(1)
  else:
    is_alone_l.append(0)

is_alone = pd.DataFrame({'IsAlone':is_alone_l})
is_alone.head()<jupyter_output><empty_output><jupyter_text>All together<jupyter_code>full_X = pd.concat([sex,df_all.Fare,family,is_alone,title,cabin,ticket,embarked,pclass],axis=1)
full_X.head()
train_X = full_X[ 0:len(train) ]
train_y = train.Survived
test_X = full_X[ len(train): ]

print (full_X.shape , train_X.shape , train_y.shape, test_X.shape)<jupyter_output>(1297, 66) (879, 66) (879,) (418, 66)<jupyter_text>Now I'm going to create a new DataFrame to compare the model results later on with the original results. P.S: I got the original data from [here](https://www.kaggle.com/tarunpaparaju/how-top-lb-got-their-score-use-titanic-to-learn)<jupyter_code>submission_df = pd.DataFrame({'PassengerId':[],'Survived':[]},dtype=int)
submission_df['PassengerId'] = test['PassengerId']
submission_df.head()<jupyter_output><empty_output><jupyter_text>Building modelNow we are good to go to build our first model. However, before we can start let's separate the data.  Define a function to fit ML algorithmsSince we will use a lot of different algorithms for the sklearn libary, they all take similiar input and give the same output. So to prevent writing code multiple times let's just define a function who does that for us:<jupyter_code>solution_data = pd.read_csv('data/test_data.csv')
solution = pd.DataFrame()
solution['PassengerId'] = solution_data['PassengerId']
solution['Survived'] = solution_data['Survived']
def predict_on_truth(model):
  liste = model.predict(test_X)
  submission_df['Survived'] = liste
  right_count = 0
  index = 0
  for i in submission_df['Survived']:
    if i == solution.iat[index,1]:
      right_count = right_count + 1
    index = index + 1  
    
  return(right_count / len(solution))    
test_size = 0.6
seed = 7
features_train, features_test, labels_train, labels_test = model_selection.train_test_split(train_X,train_y,test_size=test_size,random_state=seed)
#Function that runs the requested algorithm and returns the accuracy metrics
def fit_ml_algo(algo):
    
    cv = 10
    # One Pass
    model = algo.fit(features_train,labels_train)
    acc = predict_on_truth(model)
    
    # Cross Validation 
    test_pred = model_selection.cross_val_predict(algo, 
                                                  features_test, 
                                                  labels_test, 
                                                  cv=cv, 
                                                  n_jobs = -1)
    # Cross-validation accuracy metric
    acc_cv = round(metrics.accuracy_score(labels_test, test_pred) * 100, 2)
    
    return acc,acc_cv,model<jupyter_output><empty_output><jupyter_text>Logistic Regression<jupyter_code>start_time = time.time()
acc_log,acc_log_cv,model = fit_ml_algo(LogisticRegression())
log_time = (time.time() - start_time)
log_model = model
print("Accuracy: %s" % acc_log)
print("Accuracy CV 10-Fold: %s" % acc_log_cv)
print("Running time: %s" % datetime.timedelta(seconds=log_time))<jupyter_output>Accuracy: 0.784688995215311
Accuracy CV 10-Fold: 82.2
Running time: 0:00:00.130795<jupyter_text>K-Nearest Neighbours<jupyter_code>start_time = time.time()
acc_knn,acc_knn_cv,model = fit_ml_algo(KNeighborsClassifier())
knn_model = model
knn_time = (time.time() - start_time)
print("Accuracy: %s" % acc_knn)
print("Accuracy CV 10-Fold: %s" % acc_knn_cv)
print("Running time: %s" % datetime.timedelta(seconds=knn_time))<jupyter_output>Accuracy: 0.6985645933014354
Accuracy CV 10-Fold: 80.49
Running time: 0:00:00.099455<jupyter_text>Gaussian Naive Bayes<jupyter_code>start_time = time.time()
acc_gaussian,acc_gaussian_cv,model = fit_ml_algo(GaussianNB())
gaussian_model = model
gaussian_time = (time.time() - start_time)
print("Accuracy: %s" % acc_gaussian)
print("Accuracy CV 10-Fold: %s" % acc_gaussian_cv)
print("Running time: %s" % datetime.timedelta(seconds=gaussian_time))<jupyter_output>Accuracy: 0.423444976076555
Accuracy CV 10-Fold: 58.33
Running time: 0:00:00.063851<jupyter_text>Linear Support Vector Machines(Linear SVC)<jupyter_code>start_time = time.time()
acc_linear_svc,acc_linear_svc_cv,model = fit_ml_algo(LinearSVC())
acc_linear_svc_model = model
linear_svc_time = (time.time() - start_time)
print("Accuracy: %s" % acc_linear_svc)
print("Accuracy CV 10-Fold: %s" % acc_linear_svc_cv)
print("Running time: %s" % datetime.timedelta(seconds=linear_svc_time))<jupyter_output>Accuracy: 0.7488038277511961
Accuracy CV 10-Fold: 81.25
Running time: 0:00:00.378024<jupyter_text>Decision Tree<jupyter_code>start_time = time.time()
acc_tree,acc_tree_cv,model = fit_ml_algo(DecisionTreeClassifier())
tree_model = model
tree_time = (time.time() - start_time)
print("Accuracy: %s" % acc_tree)
print("Accuracy CV 10-Fold: %s" % acc_tree_cv)
print("Running time: %s" % datetime.timedelta(seconds=tree_time))<jupyter_output>Accuracy: 0.7655502392344498
Accuracy CV 10-Fold: 81.44
Running time: 0:00:00.079019<jupyter_text>Random Forest<jupyter_code>start_time = time.time()
acc_forest,acc_forest_cv,model = fit_ml_algo(RandomForestClassifier())
forest_model = model
forest_time = (time.time() - start_time)
print("Accuracy: %s" % acc_forest)
print("Accuracy CV 10-Fold: %s" % acc_forest_cv)
print("Running time: %s" % datetime.timedelta(seconds=forest_time))<jupyter_output>Accuracy: 0.7727272727272727
Accuracy CV 10-Fold: 79.92
Running time: 0:00:00.172041<jupyter_text>Gradient Boosting Classifier<jupyter_code>start_time = time.time()
acc_boost,acc_boost_cv,model = fit_ml_algo(GradientBoostingClassifier())
boost_model = model
boost_time = (time.time() - start_time)
print("Accuracy: %s" % acc_boost)
print("Accuracy CV 10-Fold: %s" % acc_boost_cv)
print("Running time: %s" % datetime.timedelta(seconds=boost_time))<jupyter_output>Accuracy: 0.7559808612440191
Accuracy CV 10-Fold: 82.39
Running time: 0:00:00.839038<jupyter_text>XG Boost<jupyter_code>start_time = time.time()
acc_xg,acc_xg_cv,model = fit_ml_algo(xgb.XGBClassifier())
xg_model = model
xg_time = (time.time() - start_time)
print("Accuracy: %s" % acc_xg)
print("Accuracy CV 10-Fold: %s" % acc_xg_cv)
print("Running time: %s" % datetime.timedelta(seconds=xg_time))<jupyter_output>Accuracy: 0.7583732057416268
Accuracy CV 10-Fold: 82.58
Running time: 0:00:01.043047<jupyter_text>Model ResultsWhich model had the bets cross-validation accuracy?**Note**: This is our prediction results divided by the **original** data, so this is the score we care most about. Regular accuracy scores(original data)<jupyter_code>models = pd.DataFrame({
    'Model': ['KNN', 'Logistic Regression', 'Naive Bayes', 
              'Linear SVC', 
              'Decision Tree','Random Forest','Gradient Boosting Classifier','XG Boost'],
    'Score': [
        acc_knn, 
        acc_log,  
        acc_gaussian,  
        acc_linear_svc, 
        acc_tree,
        acc_forest,
        acc_boost,
        acc_xg,
    ]})
print("---Reuglar Accuracy Scores---")
models.sort_values(by='Score', ascending=False)<jupyter_output>---Reuglar Accuracy Scores---<jupyter_text>Cross validation scores<jupyter_code>cv_models = pd.DataFrame({
    'Model': ['KNN', 'Logistic Regression', 'Naive Bayes', 
              'Linear SVC', 
              'Decision Tree','Random Forest','Gradient Boosting Classifier','XG Boost'],
    'Score': [
        acc_knn_cv,
        acc_log_cv,
        acc_gaussian_cv,
        acc_linear_svc_cv,
        acc_tree_cv,
        acc_forest_cv,
        acc_boost_cv,
        acc_xg_cv,
    ]})
print('---Cross-validation Accuracy Scores---')
cv_models.sort_values(by='Score', ascending=False)<jupyter_output>---Cross-validation Accuracy Scores---<jupyter_text>SubmissionNow we are ready to do our kaggle submission, for that we just create a new dataframe with the **PassengerID** and **Survived/not Survived**.<jupyter_code>liste = log_model.predict(test_X)

submission_df['Survived'] = liste
submission_df.head()
submission_df.to_csv('data/submission.csv',index=False)
print('Submission ready!')
submissions_check = pd.read_csv("data/submission.csv")
submissions_check.head()
right_count = 0
index = 0
for i in submission_df['Survived']:
  if i == solution.iat[index,1]:
    right_count = right_count + 1
  index = index + 1  
    
print(right_count / len(solution))<jupyter_output>0.784688995215311
{'fraction_non_alphanumeric': 0.11639676113360324, 'fraction_numerical': 0.027477882741040637, 'mean_word_length': 3.841560798548094, 'pattern_counts': {'":': 14, '<': 180, '<?xml version=': 0, '>': 182, 'https://': 3, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 15, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '46', 'dir': 'jupyter-structured-clean-dedup', 'id': '60962', 'license': 'MIT', 'n_tokens_mistral': 9832, 'n_tokens_neox': 8929, 'n_words': 2343, 'path': 'Titanic survivor prediction.ipynb', 'repo_name': 'schaepersliam/kaggle-titanic', 'size': '26676'}