<jupyter_start><jupyter_text>We should consider dropping PassengerId, Name, Ticket Number as these won't influence survival<jupyter_code>training_set = training_set.drop(['PassengerId', 'Name', 'Ticket'], axis =1)
test_set = test_set.drop(['Name', 'Ticket'], axis=1)
print(training_set.shape)
print(test_set.shape)
training_set.dtypes<jupyter_output><empty_output><jupyter_text>Cabin, Embarked and Sex are of type object, so we should convert the categorical variables into numericalsHowever, we should first try check their null counts<jupyter_code>print("Null values in Sex:", training_set['Sex'].isnull().sum())
print("Null values in Cabin:", training_set['Cabin'].isnull().sum())
print("Null values in Embarked:", training_set['Embarked'].isnull().sum())<jupyter_output>Null values in Sex: 0
Null values in Cabin: 687
Null values in Embarked: 2<jupyter_text>We observe that 687 out of the 819 entries in Cabin are null, so we should ideally drop the column<jupyter_code>training_set = training_set.drop(['Cabin'], axis =1)
test_set = test_set.drop(['Cabin'], axis=1)
print(training_set.shape)
print(test_set.shape)
print("Training Set")
print(training_set.head())
print("Test Set")
print(test_set.head())<jupyter_output>(891, 8)
(418, 8)
Training Set
   Survived  Pclass     Sex   Age  SibSp  Parch     Fare Embarked
0         0       3    male  22.0      1      0   7.2500        S
1         1       1  female  38.0      1      0  71.2833        C
2         1       3  female  26.0      0      0   7.9250        S
3         1       1  female  35.0      1      0  53.1000        S
4         0       3    male  35.0      0      0   8.0500        S
Test Set
   PassengerId  Pclass     Sex   Age  SibSp  Parch     Fare Embarked
0          892       3    male  34.5      0      0   7.8292        Q
1          893       3  female  47.0      1      0   7.0000        S
2          894       2    male  62.0      0      0   9.6875        Q
3          895       3    male  27.0      0      0   8.6625        S
4          896       3  female  22.0      1      1  12.2875        S<jupyter_text>The Sex and Embarked attributes should now be One - hot encodedBut Embarked Attribute has 2 null values, so ideally we should fill those upWe should also check if any null values exist in the test set<jupyter_code>print("Test set null counters")
print("Null values in Sex:", test_set['Sex'].isnull().sum())
print("Null values in Embarked:", test_set['Embarked'].isnull().sum())<jupyter_output>Test set null counters
Null values in Sex: 0
Null values in Embarked: 0<jupyter_text>Cool, No null values in test set, so we should ideally check out how values for each category exist in the embarked column<jupyter_code>training_set['Embarked'].value_counts()<jupyter_output><empty_output><jupyter_text>Since S dominates, we would fill up the 2 rows with Null values with 'S'<jupyter_code>training_set['Embarked'] = training_set['Embarked'].fillna('S')
print("Null values in Embarked:", training_set['Embarked'].isnull().sum())<jupyter_output>Null values in Embarked: 0<jupyter_text>Now it's time to hot encode Sex and Embarked<jupyter_code>training_set = pd.get_dummies(training_set, columns=["Sex", "Embarked"], prefix=["Sex", "Embarked"])
training_set.head()

test_set = pd.get_dummies(test_set, columns=["Sex", "Embarked"], prefix=["Sex", "Embarked"])
test_set.head()<jupyter_output><empty_output><jupyter_text>Now we can safely remove 1 column corresponding to each attribute to address the curse of dimensionalityDeliberately removing the column with Q since it has the minimum count<jupyter_code>print("Before")
print(training_set.head())
print(test_set.head())
print("After")
training_set = training_set.drop(['Sex_female', 'Embarked_Q'], axis =1)
test_set = test_set.drop(['Sex_female', 'Embarked_Q'], axis = 1)
print(training_set.head())
print(test_set.head())
training_set.dtypes<jupyter_output><empty_output><jupyter_text>Let's check Age and Fare's correlation with survival<jupyter_code>training_set['AgeBand'] = pd.cut(training_set['Age'], 5)
training_set[['AgeBand', 'Survived']].groupby(['AgeBand'], as_index=False).mean().sort_values(by='AgeBand', ascending=True)<jupyter_output><empty_output><jupyter_text>We will create Ageband features since age is highly correlated to survival<jupyter_code>training_set.loc[training_set['Age'] <= 16, 'Age'] = 0
training_set.loc[(training_set['Age'] > 16) & (training_set['Age'] <= 32), 'Age'] = 1
training_set.loc[(training_set['Age'] > 32) & (training_set['Age'] <= 48), 'Age'] = 2
training_set.loc[(training_set['Age'] > 48) & (training_set['Age'] <= 64), 'Age'] = 3
training_set.loc[ training_set['Age'] > 64, 'Age']
training_set.head()
test_set.loc[test_set['Age'] <= 16, 'Age'] = 0
test_set.loc[(test_set['Age'] > 16) & (test_set['Age'] <= 32), 'Age'] = 1
test_set.loc[(test_set['Age'] > 32) & (test_set['Age'] <= 48), 'Age'] = 2
test_set.loc[(test_set['Age'] > 48) & (test_set['Age'] <= 64), 'Age'] = 3
test_set.loc[ test_set['Age'] > 64, 'Age']
test_set.head()
training_set = training_set.drop(['AgeBand'], axis = 1)<jupyter_output><empty_output><jupyter_text>We will similarly check for Fare but we will first replace it's null values as observed above with median<jupyter_code>training_set['Fare'].fillna(training_set['Fare'].dropna().median(), inplace=True)
training_set.head()

test_set['Fare'].fillna(test_set['Fare'].dropna().median(), inplace=True)
test_set.head()
training_set['FareBand'] = pd.qcut(training_set['Fare'], 4)
training_set[['FareBand', 'Survived']].groupby(['FareBand'], as_index=False).mean().sort_values(by='FareBand', ascending=True)
training_set.loc[ training_set['Fare'] <= 7.91, 'Fare'] = 0
training_set.loc[(training_set['Fare'] > 7.91) & (training_set['Fare'] <= 14.454), 'Fare'] = 1
training_set.loc[(training_set['Fare'] > 14.454) & (training_set['Fare'] <= 31), 'Fare']   = 2
training_set.loc[ training_set['Fare'] > 31, 'Fare'] = 3
training_set['Fare'] = training_set['Fare'].astype(int)

test_set.loc[ test_set['Fare'] <= 7.91, 'Fare'] = 0
test_set.loc[(test_set['Fare'] > 7.91) & (test_set['Fare'] <= 14.454), 'Fare'] = 1
test_set.loc[(test_set['Fare'] > 14.454) & (test_set['Fare'] <= 31), 'Fare']   = 2
test_set.loc[ test_set['Fare'] > 31, 'Fare'] = 3
test_set['Fare'] = test_set['Fare'].astype(int)
training_set = training_set.drop(['FareBand'], axis=1)
training_set['Age'].fillna(training_set['Age'].dropna().median(), inplace=True)
test_set['Age'].fillna(test_set['Age'].dropna().median(), inplace=True)<jupyter_output><empty_output><jupyter_text>We will combine the Parch and SibSp attribute to create a Family Attribute<jupyter_code>training_set['FamilySize'] = training_set['SibSp'] + training_set['Parch'] + 1
test_set['FamilySize'] = test_set['SibSp'] + test_set['Parch'] + 1
#Now I will drop the SibSp and Parch attributes
training_set = training_set.drop('SibSp', axis=1)
test_set = test_set.drop('SibSp', axis=1)
training_set = training_set.drop('Parch', axis=1)
test_set = test_set.drop('Parch', axis=1)
classVar = training_set['Survived']
data = training_set.drop('Survived', axis = 1)
X_train, X_test, y_train, y_test = train_test_split(data, classVar, test_size=0.2, random_state=42)
print("Training set has {} samples.".format(X_train.shape[0]))
print("Testing set has {} samples.".format(X_test.shape[0]))
def train_predict(learner, X_train, y_train, X_test, y_test): 
    results = {}
    learner.fit(X_train, y_train)
    preds = learner.predict(X_test)
        
    #Compute accuracy on test set
    results['acc_test'] = accuracy_score(y_test, preds)
    
    #Compute F-score on the test set
    results['f_test'] = fbeta_score(y_test, preds, beta=0.5)
        
    # Return the results
    return results<jupyter_output><empty_output><jupyter_text>We will use Naive Bayes', Linear SVM and Random Forest to check which performs best<jupyter_code>clf_NB = GaussianNB()
clf_SV = LinearSVC(random_state=0)
clf_RF = RandomForestClassifier(random_state=0)

results = {}
for clf in [clf_NB, clf_SV, clf_RF]:
    clf_name = clf.__class__.__name__
    results[clf_name] = \
    train_predict(clf, X_train, y_train, X_test, y_test)
results<jupyter_output><empty_output><jupyter_text>Random Forest seems to be performing the best, hence we will try Grid Search on it<jupyter_code>clfRBIdeal = RandomForestClassifier()

parameters = {'n_estimators': [10, 50, 100],
             'min_samples_split': [2,10,20,50],
             'max_features': ['sqrt', 'log2', 'auto']}

scorer = make_scorer(fbeta_score, beta=0.5)

grid_obj = GridSearchCV(clfRBIdeal, parameters, scoring=scorer)

grid_fit = grid_obj.fit(X_train, y_train)

best_clf = grid_fit.best_estimator_

predictions = (clfRBIdeal.fit(X_train, y_train)).predict(X_test)
best_predictions = best_clf.predict(X_test)<jupyter_output><empty_output><jupyter_text>Now, we will check the accuracies and also find the ideal set of hyper parameters<jupyter_code>print("Unoptimized model")
print("Accuracy score on testing data: {:.10f}".format(accuracy_score(y_test, predictions)))
print("F-score on testing data: {:.10f}".format(fbeta_score(y_test, predictions, beta = 0.5)))
print("Optimized Model")
print("Final accuracy score on the testing data: {:.10f}".format(accuracy_score(y_test, best_predictions)))
print("Final F-score on the testing data: {:.10f}".format(fbeta_score(y_test, best_predictions, beta = 0.5)))<jupyter_output>Unoptimized model
Accuracy score on testing data: 0.8100558659
F-score on testing data: 0.7702702703
Optimized Model
Final accuracy score on the testing data: 0.8044692737
Final F-score on the testing data: 0.7748538012<jupyter_text>The test accuracy is surprisingly lower for the best model<jupyter_code>clfRBIdeal
best_clf
test_set.head()
passengerId = test_set['PassengerId']
test_set.drop('PassengerId', axis = 1, inplace = True)
pred = best_clf.predict(test_set)
pred.shape
submission = pd.DataFrame({"PassengerId": passengerId,"Survived": pred})
submission.to_csv('titanic.csv', index=False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1417141714171417, 'fraction_numerical': 0.035703570357035704, 'mean_word_length': 5.150061500615006, 'pattern_counts': {'":': 2, '<': 74, '<?xml version=': 0, '>': 74, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '132334', 'license': 'MIT', 'n_tokens_mistral': 3809, 'n_tokens_neox': 3513, 'n_words': 842, 'path': 'titanic.ipynb', 'repo_name': 'amankedia/Kaggle', 'size': '9999'}