<jupyter_start><jupyter_text>LEADERSHIP SKILLS (MSCA 31003) File: Assignment6 Description: Modeling Assignment Author: <NAME> Date: 12/04/2021 Project Overview: Predict if a passenger survived in the Titanic or not.  Instructions to get full credit:- Understand the problem (2-3 lines)Summary: Use machine learning algorithms and variables provided, to predict if a passanger survived the Titanic. Variables will be trained and tested against the parameter "Survived."<jupyter_code>import pandas as pd<jupyter_output><empty_output><jupyter_text>- Show any data cleaning that needs to be done prior to analysis<jupyter_code>file_to_load = "titanic.csv"
titanic_df = pd.read_csv(file_to_load)
titanic_df.head()<jupyter_output><empty_output><jupyter_text>- Exploratory data analysis (EDA)<jupyter_code>import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
plt.figure(figsize=(6,6))
sns.countplot(titanic_df['Survived'])
plt.xlabel('Survived')
plt.ylabel('Count')<jupyter_output>/Users/jackiepezan/opt/anaconda3/lib/python3.8/site-packages/seaborn/_decorators.py:36: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  warnings.warn(<jupyter_text>- Check the completeness of the data<jupyter_code># Check for NaN values
titanic_df.isnull().sum()<jupyter_output><empty_output><jupyter_text>- Feature engineering  Logistic Regression - Train-Test split (Feel free to choose the split)<jupyter_code>X = titanic_df.loc[:, ['Pclass','SibSp', 'Parch']]
y = titanic_df['Survived']
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=42)
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)
from sklearn.linear_model import LogisticRegression
lr = LogisticRegression(random_state=42)
lr.fit(X_train_scaled, y_train)
y_pred_lr = lr.predict(X_test_scaled)
from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report
print("\nConfusion Matrix for Logistic Regression Model:\n\n", confusion_matrix(y_test, y_pred_lr))
print("\nClassification Report for Logistic Regression Model:\n\n", classification_report(y_test, y_pred_lr))<jupyter_output>Confusion Matrix for Logistic Regression Model:

 [[179  37]
 [ 68  73]]

Classification Report for Logistic Regression Model:

               precision    recall  f1-score   support

           0       0.72      0.83      0.77       216
           1       0.66      0.52      0.58       141

    accuracy                           0.71       357
   macro avg       0.69      0.67      0.68       357
weighted avg       0.70      0.71      0.70       357<jupyter_text>- Apply 3 supervised learning algorithms (You can apply more if you’re not satisfied with results)- Make sure you standardize the features before fitting the models  Random Forest<jupyter_code>from sklearn.ensemble import RandomForestClassifier
rf = RandomForestClassifier(random_state=42)
rf.fit(X_train, y_train)
y_pred_rf = rf.predict(X_test)
print("\nConfusion Matrix for Random Forest Model:\n\n", confusion_matrix(y_test, y_pred_rf))
print("\nClassification Report for Random Forest Model:\n\n", classification_report(y_test, y_pred_rf))<jupyter_output>Confusion Matrix for Random Forest Model:

 [[173  43]
 [ 71  70]]

Classification Report for Random Forest Model:

               precision    recall  f1-score   support

           0       0.71      0.80      0.75       216
           1       0.62      0.50      0.55       141

    accuracy                           0.68       357
   macro avg       0.66      0.65      0.65       357
weighted avg       0.67      0.68      0.67       357<jupyter_text>Gradient Boost Classifier<jupyter_code>from sklearn.ensemble import GradientBoostingClassifier
gbc = GradientBoostingClassifier()
gbc.fit(X_train, y_train)
y_pred_gbc = lr.predict(X_test_scaled)
print("\nConfusion Matrix for Gradient Boosting Classifier:\n\n", confusion_matrix(y_test, y_pred_gbc))
print("\nClassification Report for Gradient Boosting Classifier:\n\n", classification_report(y_test, y_pred_gbc))<jupyter_output>Confusion Matrix for Gradient Boosting Classifier:

 [[179  37]
 [ 68  73]]

Classification Report for Gradient Boosting Classifier:

               precision    recall  f1-score   support

           0       0.72      0.83      0.77       216
           1       0.66      0.52      0.58       141

    accuracy                           0.71       357
   macro avg       0.69      0.67      0.68       357
weighted avg       0.70      0.71      0.70       357<jupyter_text>Desicion Tree Classifier<jupyter_code>from sklearn.tree import DecisionTreeClassifier
dtc = GradientBoostingClassifier()
dtc.fit(X_train, y_train)
y_pred_dtc = lr.predict(X_test_scaled)
print("\nConfusion Matrix for Desicion Tree Classifier:\n\n", confusion_matrix(y_test, y_pred_dtc))
print("\nClassification Report for Desicion Tree Classifier:\n\n", classification_report(y_test, y_pred_dtc))<jupyter_output>Confusion Matrix for Desicion Tree Classifier:

 [[179  37]
 [ 68  73]]

Classification Report for Desicion Tree Classifier:

               precision    recall  f1-score   support

           0       0.72      0.83      0.77       216
           1       0.66      0.52      0.58       141

    accuracy                           0.71       357
   macro avg       0.69      0.67      0.68       357
weighted avg       0.70      0.71      0.70       357<jupyter_text>- Calculate 3 metrics to evaluate each of the classification algorithms:<jupyter_code>Presicion, accuracy and recall will be considered as a metric in algorthim evaluation. Calculations completed above.<jupyter_output><empty_output><jupyter_text>- Which is your best performing model. Explain.<jupyter_code>In general, logistic regression performs better when the number of "noise" variables is less than or equal to the number of explanatory variables and random forest has a higher true and false positive rate as the number of explanatory variables increases in a dataset. The same goes Gradient Boosting Classifier and Desicion Tree.<jupyter_output><empty_output><jupyter_text>- Approach for deploying the model into production (Theoretical)<jupyter_code>In theory this model would need to include additional variables to increase presicion score. Further analysis of the data could prove to increase both the current accuracy and precision score.<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07273538598839803, 'fraction_numerical': 0.04462293618920125, 'mean_word_length': 3.429512516469038, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 16, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '563404', 'license': 'MIT', 'n_tokens_mistral': 2220, 'n_tokens_neox': 2034, 'n_words': 623, 'path': 'Assignment6.ipynb', 'repo_name': 'jpezan/ml-titanic-assignment', 'size': '6733'}