<jupyter_start><jupyter_text>Titanic - Machine Learning from Disaster  Data Processing  Imports<jupyter_code># for data analysis and wrangling
import pandas as pd
import numpy as np
import random as rnd
from sklearn.preprocessing import LabelEncoder

# for visualization
import matplotlib.pyplot as plt
import seaborn as sns<jupyter_output><empty_output><jupyter_text>Acquire Data<jupyter_code>train_data = pd.read_csv('https://raw.githubusercontent.com/keivanipchihagh/Competitions/sub/Kaggle/Titanic%20-%20Machine%20Learning%20from%20Disaster/Datasets/Raw%20Datasets/train.csv')
test_data = pd.read_csv('https://raw.githubusercontent.com/keivanipchihagh/Competitions/sub/Kaggle/Titanic%20-%20Machine%20Learning%20from%20Disaster/Datasets/Raw%20Datasets/test.csv')
combined_data = [train_data, test_data]<jupyter_output><empty_output><jupyter_text>Data Processing  Analyze by describing data  **Which features are available in the dataset?***PassengerId, Survive*d, Pclass, Name, Sex, Age, SibSp, Parch, Ticket, Fare, Cabin, Embarked  **Which features are categorical? Which features are ordinal?** - Ordinal:  - Pclass (1: Upper, 2: Middle, 3: Lower)- Categorical:  - Survived (1: True, 0: False)  - Sex (Male, Female)  - Embarked (C: Cherbourg, Q: Queenstown, S: Southampton)  **Which features are numerical? Which features are not?**- Numerical features:  - Continous: *Fare, Age*  - Discrete: *PassengerId, Survived, Pclass, SibSp, Parch*- Non-Numerical features: *Name, Sex, Ticket, Cabin, Embarked*  **Which features are mixed data types?***Ticket, Cabin*  **Which features may contain errors or typos?***Name: As it contains multiple formats*  **Which features contain blank, null or empty values?**- Train data: *Age, Cabin, Embarked*- Test data: *Age, Cabin, Embarked, Fare*  **What are the data types for various features?**- PassengetId: int64- Pclass: int64- Name: object- Sex: object- Age: float64- SibSp: int64- Parch: int64- Ticket: object- Fare: float64- Cabin: object- Embarked: object  **What is the distribution of numerical feature values across the samples**- About 38% of the passengers have survived.- Very few passengers paied significantly higher fare than others.- Age mean is around 29, however a few passengers are aged (maximum age is 80).  **What is the distribution of categorical features? Are entries for a feature all unique?**- All names are across unique.- Most passengers are male.- Most passengers embarked from port S(Southampton).  Assumtions based on the data description  **Completing**- We want to complete the *Age* as it is correlated to the survival rate.- We want to complete the *Embakred* as it might also correlate with survival or another important feature.  **Correcting**- *Ticket* must be dropped because it does not have any correlation to the survival rate, and also contains lots of duplicate values.- *Cabin* may also be dropped due to its lack of entries (75% of the entries are NULL).- *PassengerId* may also be dropped as it does not seem to have correlation to the survival rate.- *Name* also does not have a currect format, so we might have to drop it.  **Creating**- We might want to create a new feature called *Family* based on *Parch* and *SibSP* members as they form a family.- We may also want to extract some information from the *Name* feature before dropping it as it might provide useful information.- *Age* can be devived into a few groups instead of being descrete. (Do 28 & 29 year-olds differ really?)- We may also want to do use the same technique with *Age* to the *Fare* feature.  Analyze by pivoting features Analysis results- Passengers with Pclass of 1 had significantly higher survival chance- Female passengers had higher change of survival- Passengers with 1 or 2 SibSp had higher change of survival- Passengers with 1-3 Parch on board where likely to have survived<jupyter_code># Passengers with Pclass of 1 had significantly higher survival chance
train_data[['Pclass', 'Survived']].groupby(['Pclass'], as_index = False).mean().sort_values(by = 'Survived', ascending = False)
# Female passengers had higher change of survival
train_data[['Sex', 'Survived']].groupby('Sex', as_index = False).mean().sort_values(by = 'Survived', ascending = False)
# Passengers with 1 or 2 SibSp had higher change of survival
train_data[['SibSp', 'Survived']].groupby('SibSp', as_index = False).mean().sort_values(by = 'Survived', ascending = False)
# Passengers with 1-3 Parch on board where likely to have survived
train_data[['Parch', 'Survived']].groupby('Parch').mean().sort_values(by = 'Survived', ascending = False)<jupyter_output><empty_output><jupyter_text>Analyze by visualizing data<jupyter_code># Passengers in Pclass 1 had the highest chance of survival
graph = sns.FacetGrid(data = train_data, col = 'Survived', row = 'Pclass')
graph.map(plt.hist, 'Age', bins = 20, alpha = 0.5)<jupyter_output><empty_output><jupyter_text>Data Wrangling, correcting, cleanup, ...<jupyter_code>combined = [train_data, test_data]<jupyter_output><empty_output><jupyter_text>Drop columns 'Ticket', 'PassengerId' and 'Cabin'<jupyter_code>PassengerId = test_data['PassengerId']

for dataset in combined_data:
  dataset.drop(columns = ['Ticket', 'Cabin'], inplace = True)
  dataset.drop(columns = ['PassengerId'], inplace = True)

train_data.head(3)<jupyter_output><empty_output><jupyter_text>Feature Engineer the 'Name' column, the safely remove the column<jupyter_code>for dataset in combined:
  dataset['Title'] = dataset['Name'].str.extract('([A-Za-z]+)\.', expand = False)
rare_titles = train_data.pivot_table(index = ['Title'], aggfunc = 'size').sort_values(ascending = False)[4:].index.to_list()
rare_dictionary = dict(zip(rare_titles, ['Rare' for i in range(len(rare_titles))]))

for dataset in combined:
  dataset['Title'] = dataset['Title'].replace(rare_dictionary)

  dataset['Title'] = dataset['Title'].replace('Mlle', 'Miss')
  dataset['Title'] = dataset['Title'].replace('Ms', 'Miss')
  dataset['Title'] = dataset['Title'].replace('Mme', 'Mrs')

train_data[['Title', 'Survived']].groupby('Title', as_index = False).mean().sort_values('Survived', ascending = False)
for dataset in combined:
  encoder = LabelEncoder()  # Define new encoder
  dataset['Title'] = encoder.fit_transform(dataset['Title'])  # Encode the feature
  dataset.drop(columns = ['Name'], inplace = True)  # Drop the un-used feature

train_data.head()<jupyter_output><empty_output><jupyter_text>Correcting the data<jupyter_code>for dataset in combined_data:
  dataset['Age'] = dataset['Age'].fillna(dataset['Age'].mean())
  dataset['Embarked'] = dataset['Embarked'].fillna('C')<jupyter_output><empty_output><jupyter_text>Encode categorical features<jupyter_code>for dataset in combined:
  dataset['Sex'] = dataset['Sex'].map({'male': 0, 'female': 1}).astype(int)
  dataset['Embarked'] = dataset['Embarked'].map({'S': 0, 'C': 1, 'Q': 2}).astype(int)

train_data.head()<jupyter_output><empty_output><jupyter_text>More feature extraction On 'Age', 'Fare'<jupyter_code>train_data['AgeBand'] = pd.cut(train_data['Age'], 7)
train_data[['AgeBand', 'Survived']].groupby('AgeBand', as_index = False).mean().sort_values(by = 'Survived', ascending = False)

for dataset in combined_data:
  encoder = LabelEncoder()  # Define new instance
  dataset['AgeBand'] = pd.cut(dataset['Age'], 7)  # Cut up the dataset
  dataset['AgeBand'] = encoder.fit_transform(dataset['AgeBand'])  # Encode

  dataset.drop(columns = ['Age'], inplace = True)  # Safely remove the 'Age'

train_data.head()
test_data['Fare'].fillna(test_data['Fare'].dropna().median(), inplace=True)

train_data['FareBand'] = pd.qcut(train_data['Fare'], 4)
# train_data[['FareBand', 'Survived']].groupby('FareBand', as_index = False).mean().sort_values(by = 'FareBand', ascending = False)

for dataset in combined_data:
  dataset.loc[dataset['Fare'] <= 7.91, 'Fare'] = 0
  dataset.loc[(dataset['Fare'] > 7.91) & (dataset['Fare'] <= 14.454), 'Fare'] = 1
  dataset.loc[(dataset['Fare'] > 14.454) & (dataset['Fare'] <= 31), 'Fare']   = 2
  dataset.loc[dataset['Fare'] > 31, 'Fare'] = 3

  dataset['Fare'] = dataset['Fare'].astype(int)   

train_data.drop(columns = ['FareBand'], inplace = True)  # Safely remove the 'Fare'
train_data.head()<jupyter_output><empty_output><jupyter_text>Feature extraction on 'Parch' and 'SibSp' columns<jupyter_code>for dataset in combined:
  dataset['FamilySize'] = dataset['SibSp'] + dataset['Parch'] + 1

train_data[['FamilySize', 'Survived']].groupby('FamilySize', as_index = False).mean().sort_values(by = 'Survived', ascending = False)
for dataset in combined:
  dataset['isAlone'] = 0
  dataset['isAlone'][dataset['FamilySize'] == 1] = 1

  # Now remove 'Parch', 'SibSp' and 'FamilySize'
  dataset.drop(columns = ['SibSp', 'Parch', 'FamilySize'], inplace = True)

# train_data[['isAlone', 'Survived']].groupby('isAlone', as_index = False).mean().sort_values(by = 'Survived', ascending = False)
train_data.head()<jupyter_output><empty_output><jupyter_text>Save Processed datasets (Optional)<jupyter_code>train_data.to_csv('train.csv')
test_data.to_csv('test.csv')<jupyter_output><empty_output><jupyter_text>Model, predict and solve  Imports<jupyter_code># machine learning libraries
from sklearn.svm import SVC, LinearSVC
from sklearn.ensemble import RandomForestClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.linear_model import Perceptron, SGDClassifier, LogisticRegression
from sklearn.tree import DecisionTreeClassifier<jupyter_output><empty_output><jupyter_text>Data preperation<jupyter_code>X_train = train_data.copy()
X_train.drop(columns = ['Survived'], inplace = True)
Y_train = train_data['Survived'].copy()
X_test = test_data.copy()<jupyter_output><empty_output><jupyter_text>Models  Logistic Regression<jupyter_code>logreg = LogisticRegression()

logreg.fit(X_train, Y_train)
logreg_Y_pred = logreg.predict(X_test)

acc_log = round(logreg.score(X_train, Y_train) * 100, 2)
print('Logistic Regression Accuracy: ', round(acc_log), '%', sep = '')<jupyter_output>Logistic Regression Accuracy: 79%<jupyter_text>Support Vector Machines<jupyter_code>svc = SVC()

svc.fit(X_train, Y_train)
svc_Y_pred = svc.predict(X_test)

acc_svc = svc.score(X_train, Y_train) * 100
print('Support Vector Machine Accuracy: ', round(acc_svc), '%', sep = '')<jupyter_output>Support Vector Machine Accuracy: 82%<jupyter_text>K-Nearest Neighbors<jupyter_code>knn = KNeighborsClassifier(n_neighbors = 3)

knn.fit(X_train, Y_train)
knn_Y_pred = knn.predict(X_test)

acc_knn = knn.score(X_train, Y_train) * 100
print('K-Nearest Neighbors Accuracy: ', round(acc_knn), '%', sep = '')<jupyter_output>K-Nearest Neighbors Accuracy: 85%<jupyter_text>Gaussian Naive Bayes<jupyter_code>gaussian = GaussianNB()

gaussian.fit(X_train, Y_train)
gaussian_Y_pred = gaussian.predict(X_test)

acc_gaussian = gaussian.score(X_train, Y_train) * 100
print('Gaussian Naive Bayes Accuracy: ', round(acc_gaussian), '%', sep = '')<jupyter_output>Gaussian Naive Bayes Accuracy: 78%<jupyter_text>Perceptron<jupyter_code>perceptron = Perceptron()

perceptron.fit(X_train, Y_train)
perceptron_Y_pred = perceptron.predict(X_test)

acc_perceptron = perceptron.score(X_train, Y_train) * 100
print('perceptron Accuracy: ', round(acc_perceptron), '%', sep = '')<jupyter_output>perceptron Accuracy: 80%<jupyter_text>Linear SVC<jupyter_code>linear_svc = LinearSVC()

linear_svc.fit(X_train, Y_train)
linear_svc_Y_pred = linear_svc.predict(X_test)

acc_linear_svc = linear_svc.score(X_train, Y_train) * 100
print('Linear SVC Accuracy: ', round(acc_linear_svc), '%', sep = '')<jupyter_output>Linear SVC Accuracy: 79%<jupyter_text>Stochastic Gradient Descent<jupyter_code>sgd = SGDClassifier()

sgd.fit(X_train, Y_train)
sgd_Y_pred = sgd.predict(X_test)

acc_sgd = sgd.score(X_train, Y_train) * 100
print('Stochastic Gradient Descent Accuracy: ', round(acc_sgd), '%', sep = '')<jupyter_output>Stochastic Gradient Descent Accuracy: 64%<jupyter_text>Decision Tree<jupyter_code>decision_tree = DecisionTreeClassifier()

decision_tree.fit(X_train, Y_train)
decision_tree_Y_pred = decision_tree.predict(X_test)

acc_decision_tree = decision_tree.score(X_train, Y_train) * 100
print('Decision Tree Accuracy: ', round(acc_decision_tree), '%', sep = '')<jupyter_output>Decision Tree Accuracy: 88%<jupyter_text>Random Forest<jupyter_code>random_forest = RandomForestClassifier(n_estimators = 100)

random_forest.fit(X_train, Y_train)
random_forest_Y_pred = random_forest.predict(X_test)

acc_random_forest = round(random_forest.score(X_train, Y_train) * 100, 2)
print('Random Forest Accuracy: ', round(acc_random_forest), '%', sep = '')<jupyter_output>Random Forest Accuracy: 88%<jupyter_text>Validations<jupyter_code>models = pd.DataFrame({
    'Model': ['Support Vector Machines', 'KNN', 'Logistic Regression', 'Random Forest', 'Naive Bayes', 'Perceptron', 'Stochastic Gradient Decent', 'Linear SVC', 'Decision Tree'],
    'Score': [acc_svc, acc_knn, acc_log, acc_random_forest, acc_gaussian, acc_perceptron, acc_sgd, acc_linear_svc, acc_decision_tree]})

models.sort_values(by = 'Score', ascending = False)<jupyter_output><empty_output><jupyter_text>Submission<jupyter_code>submission = pd.DataFrame({
    'PassengerId': PassengerId,
    'Survived': decision_tree_Y_pred
})

submission.to_csv('submission.csv', index = False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13429937257245295, 'fraction_numerical': 0.011726919629518972, 'mean_word_length': 6.913120567375887, 'pattern_counts': {'":': 0, '<': 95, '<?xml version=': 0, '>': 95, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '25', 'dir': 'jupyter-structured-clean-dedup', 'id': '471017', 'license': 'Apache-2.0', 'n_tokens_mistral': 4805, 'n_tokens_neox': 4512, 'n_words': 1125, 'path': 'Kaggle/Titanic - Machine Learning from Disaster/Models/Titanic - Machine Learning from Disaster.ipynb', 'repo_name': 'keivanipchihagh/Kaggle-Competitions', 'size': '13388'}