<jupyter_start><jupyter_text>Logistic Regression with Sklearn This notebook creates and measures a logistic regression model with sklearn.* Method: [Logistic Regression](http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html)* Dataset: Titanic  Imports<jupyter_code>import pandas as pd
import numpy as np

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.metrics import accuracy_score, classification_report

from mlxtend.evaluate import confusion_matrix
from mlxtend.plotting import plot_confusion_matrix

import seaborn as sb
import matplotlib.pyplot as plt
from pylab import rcParams

%matplotlib inline
rcParams['figure.figsize'] = 10, 8
sb.set_style('whitegrid')<jupyter_output><empty_output><jupyter_text>Load and Prepare the Data<jupyter_code>DATA_FILE = "../../data/titanic.csv"
# Import the data
columns = ['PassengerId','Survived','Pclass','Name','Sex','Age','SibSp','Parch','Ticket','Fare','Cabin','Embarked']
data = pd.read_csv(DATA_FILE)
data.head()<jupyter_output><empty_output><jupyter_text>**Variable Descriptions*** PassengerId: unique passenger id* Survived: Survival (0 = No; 1 = Yes)* Pclass: Passenger Class (1 = 1st; 2 = 2nd; 3 = 3rd)* Name: Name* Sex: Sex* Age: Age* SibSp: Number of Siblings/Spouses Aboard* Parch: Number of Parents/Children Aboard* Ticket: Ticket Number* Fare: Passenger Fare (British pound)* Cabin: Cabin* Embarked: Port of Embarkation (C = Cherbourg; Q = Queenstown; S = Southampton)  Ensure Our Dependent Variable is Binary<jupyter_code># Ensure we have a binary dependent variable. Use Seaborn for this.
sb.countplot(x='Survived',data=data, palette='hls')<jupyter_output><empty_output><jupyter_text>Handle Missing Values<jupyter_code># Check for missing values
data.isnull().sum()
# See how many records we have in the dataset
data.info()
# Remove variables that seem irrelevant to our analysis or are almost all empty
data = data.drop(['PassengerId','Name','Ticket','Cabin'], 1)
data.head()
# How is age related to class?
sb.boxplot(x='Pclass', y='Age', data=data, palette='hls')<jupyter_output><empty_output><jupyter_text>**Note**: it appears that older passengers are in 1st class and younger passengers 3rd class. There may be some correlation between these two variables.<jupyter_code># Fill in the missing age of the passenger with the mean age of the class he/she is in

# Get the means
pclass_means = data.groupby('Pclass').mean()
pmv = pclass_means['Age'].values
pclass_1_mean = float(pmv[0])
pclass_2_mean = float(pmv[1])
pclass_3_mean = float(pmv[2])

# Create a custom method to assign the age
def age_approx(cols):
    Age = cols[0]
    Pclass = cols[1]
    
    if pd.isnull(Age):
        if Pclass == 1:
            return pclass_1_mean
        elif Pclass == 2:
            return pclass_2_mean
        else:
            return pclass_3_mean
    else:
        return float(Age)
# Apply the custom function
data['Age'] = data[['Age', 'Pclass']].apply(age_approx, axis=1)

# See if we have any more null values
data.isnull().sum()
# Drop the two records where embarked is null
data.dropna(inplace=True)
data.isnull().sum()<jupyter_output><empty_output><jupyter_text>Convert categorical variables to dummy indicatorsModels like numbers, not strings.<jupyter_code># Gender
gender = pd.get_dummies(data['Sex'], drop_first=True)
gender.head()
# Embarked
embark_location = pd.get_dummies(data['Embarked'], drop_first=True)
embark_location.head()
# Drop the categorical columns and concatenate the previously created dataframes
data.drop(['Sex', 'Embarked'], axis=1, inplace=True)
data = pd.concat([data, gender, embark_location], axis=1)
data.head()<jupyter_output><empty_output><jupyter_text>Check for independence between features<jupyter_code>sb.heatmap(data.corr())
# Drop Fare and Pclass as they are not independent of each other
# This is indicated by the black square being present where Pclass and Fare intersect
data.drop(['Fare', 'Pclass'], axis=1, inplace=True)
data.head()
# Prepare the train and test sets

# X = features, y = target
arr = data.values
X = arr[:,1:7]
y = arr[:,0]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.33, random_state=42)<jupyter_output><empty_output><jupyter_text>Fit a Logistic Regression Model<jupyter_code>lr = LogisticRegression()
lr.fit(X_train, y_train)<jupyter_output><empty_output><jupyter_text>Create Predictions<jupyter_code>y_pred = lr.predict(X_test)
# Create a plot to compare actual labels (Y_test) and the predicted labels (predictions)
plt.scatter(y_test, y_pred)
plt.xlabel("Actual")
plt.ylabel("Predicted")
plt.title("Actual vs. Predicted")
plt.show()<jupyter_output><empty_output><jupyter_text>Model Evaluation  AccuracyThe accuracy score is either the fraction (default) or the count (normalize=False) of correct predictions.<jupyter_code>print("Accuracy Score: %.2f" % accuracy_score(y_test, y_pred))<jupyter_output>Accuracy Score: 0.80<jupyter_text>K-Fold Cross ValidationThis estimates the accuracy of the model by splitting the data, fitting a model and computing the score 5 consecutive times. The result is a list of the scores from each consecutive run.<jupyter_code># Get scores for 5 folds over the data
clf = LogisticRegression()
scores = cross_val_score(clf, X_train, y_train, cv=5)
print(scores)<jupyter_output>[ 0.78151261  0.79831933  0.77310924  0.78991597  0.78151261]<jupyter_text>Confusion Matrix **Confusion Matrix for Binary Label**![confusion_matrix.png](attachment:confusion_matrix.png)<jupyter_code># Plot the multi-label confusion matrix
cm = confusion_matrix(y_target=y_test, 
                      y_predicted=y_pred, 
                      binary=True)

fig, ax = plot_confusion_matrix(conf_mat=cm)
plt.title("Confusion Matrix")
plt.show()<jupyter_output><empty_output><jupyter_text>**Interpretation**: 156 and 80 are the number of correct predictions. 30 and 38 are the number of incorrect predictions.  Classification Report<jupyter_code>print(classification_report(y_test, y_pred))<jupyter_output>precision    recall  f1-score   support

        0.0       0.84      0.85      0.84       184
        1.0       0.74      0.73      0.73       110

avg / total       0.80      0.80      0.80       294
{'fraction_non_alphanumeric': 0.10694422374066423, 'fraction_numerical': 0.02240584776736056, 'mean_word_length': 5.376899696048632, 'pattern_counts': {'":': 0, '<': 50, '<?xml version=': 0, '>': 50, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '601174', 'license': 'Apache-2.0', 'n_tokens_mistral': 2175, 'n_tokens_neox': 2041, 'n_words': 580, 'path': 'code/day_6/2 - Logistic Regression with Sklearn.ipynb', 'repo_name': 'anjijava16/data-analytics-machine-learning-big-data', 'size': '6293'}