<jupyter_start><jupyter_text>**K-nearest neighbor classification**  1. Defining the Question  a) Specifying the Data Analytic Question Perform K-Nearest Neighbours Classification to determine which passengers survived or didn't survive the accident  b) Defining the Metric for Success Model will be considered a success if it achieves more than 90%accuracy, the higher the accuracy the betterThe very least(lowest acceptable accuracy) accuracy should be 80%  c) Understanding the context  The dataset contains records of people who did and didn't survive the titanic incident It contains information on the passagers i.e their names, class they ferried in, sex, age, ticket details, fare paid ...  d) Recording the Experimental Design 1.  Reading data 2.  Checking data for errors3.  Tidying dataset4.  Performing Exploratory Data Analysis5.  Perfoming predictions  e) Data Description survival:	Survival	0 = No, 1 = Yespclass:Ticket class	1 = 1st, 2 = 2nd, 3 = 3rdsibsp: Column defines family relations(sibling and spouse)parch: Column defines family relations(Parent and Child)Embarked:	Port of Embarkation(C = Cherbourg, Q = Queenstown, S = Southampton)  2. Reading the Data<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
dataset = pd.read_csv("/content/train_set.csv")<jupyter_output><empty_output><jupyter_text>3. Checking the Data<jupyter_code># Determining the shape of dataset
dataset.shape

#there are 891 rows and 12 columns
# Previewing the top of our dataset
dataset.head()
# Previewing the bottom of our dataset
dataset.tail()
# Checking datatype
dataset.dtypes
#checking for unique values
dataset['Survived'].unique()<jupyter_output><empty_output><jupyter_text>if passanger survived or not,  0 represents No and 1 represents yes<jupyter_code>#checking for unique values
dataset['Pclass'].unique()

#there are only 3 classes
#checking for unique values
dataset['Embarked'].unique()<jupyter_output><empty_output><jupyter_text>4. Tidying the Dataset<jupyter_code>#Dropping irrelevant column
dataset.drop(['PassengerId', 'Ticket'], axis=1, inplace=True)
dataset.head(2)
#Dropping irrelevant column
dataset.drop(['Name'], axis=1, inplace=True)
dataset.head(2)
#checking for duplicates
dataset.duplicated().sum()<jupyter_output><empty_output><jupyter_text>No presence of duplicates<jupyter_code># Identifying the Missing Data
dataset.isnull().sum()
#checking percentage of missing values
missing= 100 * dataset.isnull().sum() / len(dataset)
print(missing)
#dropping the Cabin 
dataset.drop(['Cabin'], axis=1, inplace=True)
dataset.head(2)<jupyter_output><empty_output><jupyter_text>Column was dropped due to a high percentage of missing data of more than 75%<jupyter_code>#replacing the null values in embarked column with forward filing

dataset['Embarked'].fillna(method='ffill', inplace=True)
dataset['Age'].fillna(method='ffill', inplace=True)<jupyter_output><empty_output><jupyter_text>used forward filing since it will not disrupt the data<jupyter_code>#Checking if missing values have been replaced
dataset.isnull().sum()
# Checking for Outliers
col=['Survived','Pclass','Age','SibSp','Parch','Fare']
for i in col:
  fig=plt.figure(figsize = (10,7))
  plt.boxplot(dataset[i])
  plt.title(i)
  plt.show()
#Removing outliers

Q1 = dataset.quantile(0.25)
Q3 = dataset.quantile(0.75)
IQR = Q3 - Q1

# Removing outliers
 
df = dataset[~((dataset < (Q1 - 1.5 * IQR)) | (dataset > (Q3 + 1.5 * IQR))).any(axis=1)]

# Printing the shape of our new dataset
print(df.shape)                                   #Our new dataset has 600 rows and 8 columns<jupyter_output>(600, 8)<jupyter_text>We will remove outliers since KNN is adversely affected by the presence of outliers<jupyter_code>df.isnull().sum()<jupyter_output><empty_output><jupyter_text>Our Clean dataset has 600 rows and 8 columns, no missing value nor duplicates  5. Exploratory Analysis  ***Univariate Analysis***<jupyter_code>#frequency table for age variable
dataset.Pclass.value_counts()<jupyter_output><empty_output><jupyter_text>most of the passangers boarded the third class<jupyter_code>#frequency table for age variable
dataset.Sex.value_counts()<jupyter_output><empty_output><jupyter_text>most of the passangers were male<jupyter_code>dataset.head(2)
# Finding the measures of central tendencies in the numerical columns

numerical_columns =['Survived','Pclass','Age','SibSp','Parch','Fare']

for column in numerical_columns:
  print('\n')
  print(column + ', Mean: '+ str(dataset[column].mean()))
  print(column + ', Mode: '+ str(dataset[column].mode()))
  print(column + ', Median: '+ str(dataset[column].median()))
  print(column + ', Standard deviation: '+ str(dataset[column].std()))
  print(column + ', Variance: '+ str(dataset[column].var()))
  print(column + ', Skewness: '+ str(dataset[column].skew()))
  print(column + ', Kurtosis: '+ str(dataset[column].kurt()))
#Bargraph showcasing 
dataset['SibSp'].value_counts().head().plot(kind= 'barh');

plt.xlabel("Count")
plt.ylabel("SibSp")

# Giving a title to our Bar chart
plt.title("Presence of Sibling\Spouse relation");<jupyter_output><empty_output><jupyter_text>Most of the passangers had no record of having siblings or spouse relation<jupyter_code>#visualization of age using displot
plt.figure(figsize=(8, 8))
sns.displot(data=df, x='Age');
plt.show();<jupyter_output><empty_output><jupyter_text>Most of the passangers are between the age of 20 to 30<jupyter_code>#visualizing our survivors using countplot
plt.figure(figsize=(8, 8))
sns.countplot(data=df, x='Survived')
plt.title('Comparison of survivors vs dead passangers')
plt.show()<jupyter_output><empty_output><jupyter_text>Most the passangers didn't survive   ***Bivariate analysis***<jupyter_code># Relationship between age of the passenger and the amount of fare paid
plt.figure(figsize=(8, 8))
sns.scatterplot(data=df, x='Age', y='Fare');
#comparing the gender with most survivors and their passanger class

plt.figure(figsize=(8, 8))
sns.barplot(data=df, x='Sex', y='Survived', hue='Pclass')
plt.title('Comparison of survivors by sex and class')
plt.show()<jupyter_output><empty_output><jupyter_text>Most of the male passanger borded the first class and most of the females also boarded the first class<jupyter_code>#convert categorical data to numerical
from sklearn.preprocessing import LabelEncoder
en= LabelEncoder()

df['Sex'] = en.fit_transform(df['Sex'])
df['Embarked'] = en.fit_transform(df['Embarked'])
df.head()<jupyter_output><empty_output><jupyter_text>Sex = 1 is Male, 0 is female<jupyter_code>#correlation using a heatmap
fig,ax = plt.subplots(figsize=(10, 10))
sns.heatmap(df.corr(),annot=True)
plt.show()<jupyter_output><empty_output><jupyter_text>The relationship between fare and SibSp variable which is 0.33 shows that it is a postive correlation ie if one increases the other also increases.The relationship between fare and Pclass is -0.68 shows a negative correlation ie if one increases the other decreases.<jupyter_code>correlations = df.corr()

correlations<jupyter_output><empty_output><jupyter_text>Our variables have a low correlation thus no need to drop anyResearch was done as to why the Parch Variable showcases NaN and results state that it is because the variable has no correlation with other variables   6. Modeling  Using 80 - 20 split<jupyter_code>from sklearn.model_selection import train_test_split, GridSearchCV, KFold
from sklearn.preprocessing import Normalizer, StandardScaler, LabelEncoder

from sklearn.neighbors import KNeighborsClassifier, KNeighborsRegressor
from sklearn.naive_bayes import MultinomialNB, GaussianNB, BernoulliNB
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report
#Splitting independent and independent variables

X = df[['Pclass','Sex','Age','SibSp','Parch','Fare','Embarked']].values

y = df['Survived'].values
#splitting to train and test 80-20

X_train,X_test, Y_train,Y_test= train_test_split(X,y, test_size = 0.2, random_state=0)
#normalizing data using feature scaling (model optimization)
scaler = StandardScaler()
scaler.fit(X_train)

X_train = scaler.transform(X_train)
X_test = scaler.transform(X_test)
#finding the best k_value using GrindSearchCV
paramas = {'n_neighbors':[5,6,7,8,9,10,11,12,13,14,15,16,17,18,20]}
knn = KNeighborsClassifier()

model = GridSearchCV(knn, paramas, cv=5)
model.fit(X_train,Y_train)
model.best_params_<jupyter_output><empty_output><jupyter_text>K = 12 we will be using this for our prediction<jupyter_code>#Training dataset
classifier = KNeighborsClassifier(n_neighbors=12)
classifier.fit(X_train, Y_train)
#Making predictions
y_pred = classifier.predict(X_test)
#getting Accuracy, Confusion matrix and classification report
print('Accuracy of our 80-20 model is : ', accuracy_score(Y_test, y_pred))

print('\n*************************************************')

print('\nConfusion matrix : ', confusion_matrix(Y_test, y_pred))

print('\n*************************************************')

print('\nClassification Report : ', classification_report(Y_test, y_pred))<jupyter_output>Accuracy of our 80-20 model is :  0.8416666666666667

*************************************************

Confusion matrix :  [[75  5]
 [14 26]]

*************************************************

Classification Report :                precision    recall  f1-score   support

           0       0.84      0.94      0.89        80
           1       0.84      0.65      0.73        40

    accuracy                           0.84       120
   macro avg       0.84      0.79      0.81       120
weighted avg       0.84      0.84      0.84       120<jupyter_text>The accuracy for the first model using 80% as the training set and 20% as a test set is 84%  Using 70 - 30 split<jupyter_code>#splitting to train and test 70-30

X2_train,X2_test, Y2_train,Y2_test= train_test_split(X,y, test_size = 0.3, random_state=0)
#Training dataset
classifier = KNeighborsClassifier(n_neighbors=12)
classifier.fit(X2_train, Y2_train)
#Making predictions
y_pred2 = classifier.predict(X2_test)
#getting Accuracy, Confusion matrix and classification report
print('Accuracy of our 70-30 model is : ', accuracy_score(Y2_test, y_pred2))

print('\n*************************************************')

print('\nConfusion matrix : ', confusion_matrix(Y2_test, y_pred2))

print('\n*************************************************')

print('\nClassification Report : ', classification_report(Y2_test, y_pred2))<jupyter_output>Accuracy of our 70-30 model is :  0.6944444444444444

*************************************************

Confusion matrix :  [[115  11]
 [ 44  10]]

*************************************************

Classification Report :                precision    recall  f1-score   support

           0       0.72      0.91      0.81       126
           1       0.48      0.19      0.27        54

    accuracy                           0.69       180
   macro avg       0.60      0.55      0.54       180
weighted avg       0.65      0.69      0.64       180<jupyter_text>The accuracy for the second model using 70% as the training set and 30% as a test set is 69%  Using 60 - 40 Split<jupyter_code>#splitting to train and test 60-40

X3_train,X3_test, Y3_train,Y3_test= train_test_split(X,y, test_size = 0.4, random_state=0)
#Training dataset
classifier = KNeighborsClassifier(n_neighbors=12)
classifier.fit(X3_train, Y3_train)
#Making predictions
y_pred3 = classifier.predict(X3_test)
#getting Accuracy, Confusion matrix and classification report
print('Accuracy of our 60-40 model is : ', accuracy_score(Y3_test, y_pred3))

print('\n*************************************************')

print('\nConfusion matrix : ', confusion_matrix(Y3_test, y_pred3))

print('\n*************************************************')

print('\nClassification Report : ', classification_report(Y3_test, y_pred3))<jupyter_output>Accuracy of our 60-40 model is :  0.7166666666666667

*************************************************

Confusion matrix :  [[157  10]
 [ 58  15]]

*************************************************

Classification Report :                precision    recall  f1-score   support

           0       0.73      0.94      0.82       167
           1       0.60      0.21      0.31        73

    accuracy                           0.72       240
   macro avg       0.67      0.57      0.56       240
weighted avg       0.69      0.72      0.67       240
{'fraction_non_alphanumeric': 0.146268896751367, 'fraction_numerical': 0.0380347378578321, 'mean_word_length': 4.979326923076923, 'pattern_counts': {'":': 0, '<': 82, '<?xml version=': 0, '>': 82, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 3, 'word_list_counts': {'cursed_substrings.json': 14, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '21', 'dir': 'jupyter-structured-clean-dedup', 'id': '435797', 'license': 'MIT', 'n_tokens_mistral': 4229, 'n_tokens_neox': 3842, 'n_words': 1102, 'path': 'KNN___IP.ipynb', 'repo_name': '01DaisyLynn/KNN-NAIVEBAYES', 'size': '12436'}