<jupyter_start><jupyter_text>TITANIC SURVIVAL PREDICTION- KNN Algorithm  Defining the question - To predict whether an individual survived based on the features in the dataset Metric for success - This project will be considered successful if we are able to obtain a high accuracy score in our model (KNN) when predicting chances of survival of an individual in the titanic. Understanding the context - The Titanic was a luxury ship, which at its time, was the largest ship afloat.- It was the second of three Olympic-class ocean liners operated by the White Star Line.- On April 15, 1912, the Titanic embarked on its maiden voyage and sank after colliding with an iceberg, killing 1502 out of 2224 passengers and crew. The tragedy is considered one of the most infamous shipwrecks in history and led to better safety guidelines for ships.- We shall analyse the data on passengers aboard the Titanic to build a model that can predict whether a passenger survived or not.- We have two data sets. One for training (train.csv) containing survival and death information that we will use to train our model. One for testing (test.csv), without survival and death information, that we will use to test our models. Experimental design - Defining the Question- Reading the Data.- Checking the Data.- Data Cleaning- Performing EDA- Building thePrediction Model<jupyter_code># Import Libraries

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import warnings
from imblearn.over_sampling import SMOTE
from sklearn.preprocessing import RobustScaler, StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report, roc_auc_score, f1_score

# Set global parameters
%matplotlib inline
sns.set()
plt.rcParams['figure.figsize'] = (10.0, 8.0)
warnings.filterwarnings('ignore')
# reading our csv files
train = pd.read_csv('/content/train (5).csv')
test = pd.read_csv('/content/test (1).csv')
# checking the top of our train dataset
train.head()
# checking the bottom of train our dataset
train.tail()
# checking the data types
train.info()
# Summary Statistics of the data
train.describe().T<jupyter_output><empty_output><jupyter_text>Data Cleaning<jupyter_code># making our columns uniform
train.columns = train.columns.str.lower()
test.columns = test.columns.str.lower()
# Checking for null values
train.isnull().sum()
# Dropping cabin column
train.drop( columns=['cabin'],inplace = True)
test.drop( columns=['cabin'],inplace = True)
# Replacing the null values in age column with the mean
train['age'] = train['age'].fillna(train['age'].mean())
# dealing with null values in embarked column
# There are only 2 null values. We will drop this 2 observations
train.dropna(inplace=True)
test.dropna(inplace=True)
# Checking for outliers
# Checking for outliers using boxplots
train.boxplot(figsize=(15, 8), fontsize=10)

# print size of the data with outliers present
print(train.shape)<jupyter_output>(889, 11)<jupyter_text>- The age column and fare have the most outlier values- We will ignore them and proceed with the analysis   EDA<jupyter_code># Checking the frequency of the survived column
train.survived.value_counts(normalize=True)*100<jupyter_output><empty_output><jupyter_text>- 59.6% of the passengers on board did not survive- 40.4% of the passengers on board survived<jupyter_code>#checking what gender most passengerrs were
print('\n\nSex Ratio of individuals')
print(train['sex'].value_counts(normalize=True))<jupyter_output>Sex Ratio of individuals
male      0.649044
female    0.350956
Name: sex, dtype: float64<jupyter_text>- 63.6% of the passengers were men- 36% of the passengers were ladies<jupyter_code>#Checking for survival rates by gender
# we know women and children were given the first priority when being evacuated from the boat

#we will filter the dataset to get records of only those who survived
surv=train[train['survived']==1]

print('Survival Rate by Sex')
surv['survived'].groupby(surv['sex']).value_counts()<jupyter_output>Survival Rate by Sex<jupyter_text>- The number of women who had boarded the titanic were less than that of men.- Despite that fact, most women survived than men<jupyter_code># plotting the passenger age historgram.
hist = train['age'].hist(bins=30)<jupyter_output><empty_output><jupyter_text>- most of the passengers were young compare to the elderly<jupyter_code># survival rate by age
surv_age=surv['age'].hist(bins=30)<jupyter_output><empty_output><jupyter_text>- most of the young people survived- all the elderly people also survived- Most of the passengers who survived were in their twenties and thirties- None of the passengers in their seventies survived<jupyter_code># plotting count plots of our features
fig, axes = plt.subplots(2, 4, figsize=(16, 10))
sns.countplot('survived',data=train,ax=axes[0,0])
sns.countplot('pclass',data=train,ax=axes[0,1])
sns.countplot('sex',data=train,ax=axes[0,2])
sns.countplot('sibsp',data=train,ax=axes[0,3])
sns.countplot('parch',data=train,ax=axes[1,0])
sns.countplot('embarked',data=train,ax=axes[1,1])
sns.distplot(train['fare'], kde=True,ax=axes[1,2])
sns.distplot(train['age'],kde=True,ax=axes[1,3])
# plotting pairplots
plt.figure(dpi= 80)
sns.pairplot(train[['age','fare']], kind="scatter", plot_kws=dict(s=80, edgecolor="white", linewidth=2.5))
plt.show()
# correlation heatmap 
colormap = plt.cm.Blues
plt.figure(figsize=(14,10))
sns.heatmap(train.corr(), cmap=colormap, annot=True, linewidths=0.2)<jupyter_output><empty_output><jupyter_text>- From the correlation map, we can see that p-class and fare are the columns with the highest correlation coefficient with our target variable<jupyter_code>explode = [0,0,0]
p = surv.groupby('pclass')['survived'].value_counts()

labels = surv['pclass'].unique()

plt.title('A pie chart showing the % of pclass who survived.')
plt.pie(p,labels=labels,explode=explode,autopct='%1.1f%%',startangle=90,shadow=True);<jupyter_output><empty_output><jupyter_text>- Most of the passengers who survived were in the first class of the titanic.<jupyter_code># Comparing class and fare
sns.catplot(x='pclass', y='fare', data=train)<jupyter_output><empty_output><jupyter_text>- People in the first class paid high fares than those in the second and third class of the titanic<jupyter_code>#Comparing sex and age and pclass
sns.catplot(x='sex', y='age', data=train, kind='box', hue='pclass')<jupyter_output><empty_output><jupyter_text>- Majority of the elderly passengers were in the first class of the titanic<jupyter_code># What about the family's size? Would a larger family stand a better chance of survival?
train['family_members'] = train['parch'] + train['sibsp']
test['family_members'] = test['parch'] + test['sibsp']
# Lets check family size
train['survived'].groupby(train['family_members']).mean().plot(kind='bar')<jupyter_output><empty_output><jupyter_text>- small families had higher survival rates<jupyter_code># dropping columns we will not need 
train.drop(['name','sibsp','parch','passengerid','ticket'],axis=1,inplace=True)
test.drop(['name','sibsp','parch','passengerid','ticket'],axis=1,inplace=True)
train.columns
# Encode sex and embarked features
train['sex'] = train.sex.map({'male':0, 'female':1})
train['embarked'] = train.embarked.map({'C':0,'S':1,'Q':2})

test['sex'] = test.sex.map({'male':0, 'female':1})
test['embarked'] = test.embarked.map({'C':0,'S':1,'Q':2})
train.head()<jupyter_output><empty_output><jupyter_text>Modelling  Baseline<jupyter_code># Defining the predictor and target variables in our dataset.
#
X = train.drop(['survived'],axis=1)
y = train['survived']

# One hot encoding our data for the categorical variables
#
X = pd.get_dummies(X, drop_first=True)

# Splitting our dataset
#
X_train,X_test,y_train,y_test = train_test_split(X,y,random_state=0,test_size=0.2)

# Scaling the predictor variables
#
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.fit_transform(X_test)

# Fitting the data in a logistic regression
#
from sklearn.linear_model import LogisticRegression
regressor = LogisticRegression()
regressor.fit(X_train,y_train)

# Predicting classification using the test set
#
y_predict = regressor.predict(X_test)

# Getting the metrics for assessing our baseline model
#
print(confusion_matrix(y_test,y_predict))
print(classification_report(y_test,y_predict))<jupyter_output>[[87 18]
 [31 42]]
              precision    recall  f1-score   support

           0       0.74      0.83      0.78       105
           1       0.70      0.58      0.63        73

    accuracy                           0.72       178
   macro avg       0.72      0.70      0.71       178
weighted avg       0.72      0.72      0.72       178<jupyter_text>Our model's accuracy is 72%, which is not high enough for us.  K Nearest Neighbours<jupyter_code># Defining the predictor variables and target column
#
X = train.drop(['survived'],axis=1)
y = train['survived']
scaler = StandardScaler()
# Standardizing our numerical columns which are the age and fare columns
#
X[['age','fare']] = scaler.fit_transform(X[['age','fare']])

# One hot encoding any categorical columns in our dataset
#
X = pd.get_dummies(X, drop_first=True)

# Splitting our dataset into the test and train 
#
X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.2,random_state=101)
 
# Choosing an odd number 3 to help us classify our data
#
classifier = KNeighborsClassifier(n_neighbors=3)
classifier.fit(X_train,y_train)
# Predict new values using our test data
#
y_pred = classifier.predict(X_test)

# Printing out the various metrics
#
print(confusion_matrix(y_test,y_pred))
print(classification_report(y_test,y_pred))<jupyter_output>[[89 18]
 [19 52]]
              precision    recall  f1-score   support

           0       0.82      0.83      0.83       107
           1       0.74      0.73      0.74        71

    accuracy                           0.79       178
   macro avg       0.78      0.78      0.78       178
weighted avg       0.79      0.79      0.79       178<jupyter_text>- When using KNN classifier, the accuracy increased from 72% to 79%<jupyter_code># Hyperparameter tuning
#  Defining the hyper parameters for the Knearest Neighbors Classifier
#
leaf_size = list(range(1,50))
n_neighbors = list(range(1,30))
p = [1,2]
metric = ['manhattan', 'euclidean', 'minkowski']

# Creating a dictionary with the hyperparameters
#
hyperparameters = dict(leaf_size = leaf_size, n_neighbors = n_neighbors, p=p,metric = metric)
# Used GridSearch to look for the best parameters
classifier = KNeighborsClassifier()

clf = GridSearchCV(classifier, hyperparameters,cv=10)
# Fitting the train data into our model
#
clf.fit(X_train,y_train)
# Printing out the best parameters
#
clf.best_params_
# Fitting the parameters into our model
#
classifier = KNeighborsClassifier(leaf_size=10,n_neighbors=22,p=1,metric='manhattan')
classifier.fit(X_train,y_train)
# Predicting values using our test set
#
y_predict = classifier.predict(X_test)

# Printing out the metrics for assessing the model's perfromance
#
print(confusion_matrix(y_test,y_pred))
print(classification_report(y_test,y_pred))
# Defining the list of test sizes we will use for the assessment
#
test_size = [0.1, 0.2, 0.3, 0.4, 0.5]

from sklearn import metrics
# Using a for loop to split the dataset, fit it with the optimal parameters then get the accuracy score.
#

for test in test_size:
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = test, random_state = 12)
    
    # Fitting to the classifier
    classifier = KNeighborsClassifier(leaf_size=1,n_neighbors=29,p=1,metric='manhattan')
    classifier.fit(X_train, y_train)
    
    y_pred = classifier.predict(X_test)

    print("Test size {} has accuracy score:".format(test), (metrics.accuracy_score(y_test, y_pred)*100))<jupyter_output>Test size 0.1 has accuracy score: 87.64044943820225
Test size 0.2 has accuracy score: 83.14606741573034
Test size 0.3 has accuracy score: 82.02247191011236
Test size 0.4 has accuracy score: 80.6179775280899
Test size 0.5 has accuracy score: 78.42696629213484
{'fraction_non_alphanumeric': 0.10462940362867139, 'fraction_numerical': 0.0318118948824343, 'mean_word_length': 5.179989944695827, 'pattern_counts': {'":': 0, '<': 59, '<?xml version=': 0, '>': 59, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '304989', 'license': 'MIT', 'n_tokens_mistral': 4259, 'n_tokens_neox': 3916, 'n_words': 1218, 'path': 'titanic.ipynb', 'repo_name': 'GeorgeKMaina/knn-naive-bayes', 'size': '12291'}