<jupyter_start><jupyter_text>Logistic Regression with Python<jupyter_code>## Semicleaned Dataset from Kaggle<jupyter_output><empty_output><jupyter_text>Import LibrariesLet's import some libraries to get started!<jupyter_code>import pandas as pd
import numpy as np 
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline<jupyter_output><empty_output><jupyter_text>The DataLet's start by reading in the titanic_train.csv file into a pandas dataframe.<jupyter_code>train = pd.read_csv('titanic_train.csv')
# Reading titanic_train file as dataframe
import warnings
warnings.filterwarnings("ignore")<jupyter_output><empty_output><jupyter_text>Data Dictionary| **Variable** | **Definition**                             | **Key**                                        || :----------- | :----------------------------------------- | :--------------------------------------------- || survival     | Survival                                   | 0 = No, 1 = Yes                                || pclass       | Ticket class                               | 1 = 1st, 2 = 2nd, 3 = 3rd                      || sex          | Sex                                        |                                                || Age          | Age in years                               |                                                || sibsp        |  of siblings / spouses aboard the Titanic |                                                || parch        |  of parents / children aboard the Titanic |                                                || ticket       | Ticket number                              |                                                || fare         | Passenger fare                             |                                                || cabin        | Cabin number                               |                                                || embarked     | Port of Embarkation                        | C = Cherbourg, Q = Queenstown, S = Southampton | Variable Notes**pclass**: A proxy for socio-economic status (SES)1st = Upper2nd = Middle3rd = Lower**age**: Age is fractional if less than 1. If the age is estimated, is it in the form of xx.5**sibsp**: The dataset defines family relations in this way...Sibling = brother, sister, stepbrother, stepsisterSpouse = husband, wife (mistresses and fiancés were ignored)**parch**: The dataset defines family relations in this way...Parent = mother, fatherChild = daughter, son, stepdaughter, stepsonSome children travelled only with a nanny, therefore parch=0 for them.<jupyter_code>train.head()<jupyter_output><empty_output><jupyter_text>Exploratory Data AnalysisLet's begin some exploratory data analysis! We'll start by checking out missing data! Missing DataWe can use seaborn to create a simple heatmap to see where we are missing data!<jupyter_code>sns.heatmap(train.isnull(),yticklabels=False,cbar=False,cmap="viridis")
#train.isnull()
print((train.isnull()).groupby('Age').count())
print((train.isnull()).groupby('Cabin').count())
print((train.isnull()).groupby('Embarked').count())<jupyter_output><empty_output><jupyter_text>- This creates a heatmap of boolean values True/False we get from train.isnull(), every yellow dash stands for a true point, that is the point where null is present there we have true. From this we can infer that lots of age information is missing and the information missing the most is cabin information and very few in embarked. Roughly about 20% of age data is missing.- As age data missing proportionately is likely small enough for us to be able to replace it in some manner of computation, we can use knowledge from other columns to fill in the missing values of age column in a reasonable manner.- However, on seeing cabin column we are missing too much of the data to do something useful out of it. We might as well drop it.<jupyter_code>sns.set_style('whitegrid')
# Getting an idea about ratio of actual target variables of survival on train data
sns.countplot(x='Survived',data=train)
# Roughly speaking closer to 550 people perished and 300 something people survived.
#  Seeing survival as per gender
sns.countplot(x='Survived',hue="Sex",data=train,palette="RdBu_r")
# Looks like Jack did made a sacrifice. Clearly more women survived when compared to men onboard.
# We will see more about these when we analyse factors which play a crucial role in someone's survival.
# Visualising as per Passenger class to compare people of which class survived more
sns.set_style('whitegrid')
sns.countplot(x='Survived',hue='Pclass',data=train,palette='rainbow')
# We can infer that people who perished among those there are more probability of those belongging to class 3.
# People who survived had more probability of being from class 1 or 2.
# Let's get an idea about age of people onboard : For this we can do a distribution type plot
sns.distplot(train['Age'].dropna(),kde=True,color='darkred',bins=30)
#  What we can infer is that this appears to be bimodal distribution, there are quite a few passengers in age range of 0-10,
#  after that there is an average age between 20-30 and as older you get number of passengers decreases onboard.
# To get basic idea about columns
train.info()
# We have seen Survived column, Sex, Age, Pclass column already, let's check SibSp - Siblings/Spouse onboard column
sns.countplot(x='SibSp',data=train)
# Looking at this plot we can see that around 600 had 0 siblings or spouse onboard, closer to 200 people had 1 of the sibling
# or spouse onboard. Very few numbers of siblings/spouse for other 91 passengers in train dataset.
# Thus, there were a lot of single people onboard.
sns.countplot(x='SibSp',data=train,hue='Survived',palette="RdBu_r")
# Comparing amongst SibSp column who survived or not we see that survival factor is more for people who were not travelling 
# alone.<jupyter_output><empty_output><jupyter_text>- We have explored the data, got some idea about nature of the dataset, how columns realte with survival rate in some cases, now we can move towards Data Cleaning and Building a logistic regression model. ___ Data CleaningWe want to fill in missing age data instead of just dropping the missing age data rows. One way to do this is by filling in the mean age of all the passengers (imputation).However we can be smarter about this and check the average age by passenger class. For example:<jupyter_code>plt.figure(figsize=(12, 7))
sns.boxplot(x='Pclass',y='Age',data=train)
# We can see that upon separation by class the wealthier passengers in the first and second class tend to be actually a bit more
# older than the passengers of third class. We can use these average age vallues to impute the age based off of passenger class.<jupyter_output><empty_output><jupyter_text>We can see the wealthier passengers in the higher classes tend to be older, which makes sense. We'll use these average age values to impute based on Pclass for Age.<jupyter_code>#  We may create an entire model to predict age based off of other features.
def impute_age(cols):
    Age = cols[0] # First item in cols
    Pclass = cols[1] # Second item
    
    if pd.isnull(Age):
        
        if Pclass == 1:
            return 37
        elif Pclass == 2:
            return 29
        else: # Not in first or second class means it is third class
            return 24
    else:
        #If age is null then we return some int based upon average age of that Pclass but if it is not then we do following :
        return Age
train['Age'] = train[['Age','Pclass']].apply(impute_age,axis=1)<jupyter_output><empty_output><jupyter_text>What I did above ?- Created a function that takes in an argument cols, based off of index values we grab Age and Pclass, as we are using .apply() on Age and Pclass columns then we search for condition where pd.isnull(Age) is true then we look for the Pclass for that entry, and we return the average age of that class based off of Pclass and so on for second and third class. - If it is not null then we know the age and return the actual age.- axis = 1 for applying on columns. Checking Heatmap<jupyter_code>sns.heatmap(train.isnull(),yticklabels=False,cbar=False,cmap='viridis')
# This means that now we do not have any missing information for age column. We have successfully put in age values based off
# of people's class's average./
# About cabin column, as there are too many missing points in here it is easier to drop the cabin column.
train.drop("Cabin",axis=1, inplace=True)
train.head()
# We no longer have cabin column and now upon running heatmap again
sns.heatmap(train.isnull(),yticklabels=False,cbar=False,cmap='viridis')
#As embarked has only a row or two of missing values we drop it
train.dropna(inplace=True)
sns.heatmap(train.isnull(),cmap='viridis',yticklabels=False,cbar=False)
train.head()<jupyter_output><empty_output><jupyter_text>- Now we do not have any missing values. We filled in some of the missing values and dropped a very few. Now data is clean.  Converting Categorical Features We'll need to convert categorical features to dummy variables using pandas! Otherwise our machine learning algorithm won't be able to directly take in those features as inputs. What we mean to say that for columns like Sex we have male or female, for ML algorithm to understand this we have to assign them a number say 0 for male and 1 for female, this is called creating a dummy variable and we would be doing the same for Embarked columns like Q,S,C.<jupyter_code># To do this we use get_dummies method() of pandas which converts categorical variables into dummy or indicator variables.

dummy = pd.get_dummies(train['Sex']) # Passing in the column we want to convert and gives us a dataframe with col for every category.
# Here 0 in female column means there would be a 1 in male colummn for that entry as it can be male or female here.
# 1 in female column would mean this entry was a female and the male corresponding column would show 0.

dummy.head()

# The issue with what we get here is that in the one column is a perfect predictor of the other column, meaning if our
# ML algorithm gets fed the columns then ML algorithm will immediately know that if it's 0 at female then it 
# will predict perfectly that it is 1 at male. This is going to be an issue called multi-collinearity. It will mess up the
# algorithm as bunch of columns will be perfect predictors of other columns. To avoid this we type drop_first = True
sex = pd.get_dummies(train['Sex'],drop_first=True)
sex.head()
embark = pd.get_dummies(train['Embarked'],drop_first=True)
embark.head()
# We see that dropping C would result in current indicators being present of Q and S and as these are not perfect indicators
# of each other.
train = pd.concat([train,sex,embark],axis=1) # To add new indicator variable or dummy to dataframe via concatenation
train.head()
# Now we can drop columns like Name, Ticket, Embarked and Sex
train.drop(['Sex','Embarked','Name','Ticket','PassengerId'],axis=1,inplace=True)
train.head()
train = pd.concat([train,sex,embark],axis=1)
train.head()<jupyter_output><empty_output><jupyter_text>Great! Our data is ready for our model! Building a Logistic Regression modelLet's start by splitting our data into a training set and test set (there is another test.csv file that you can play around with in case you want to use all this data for training). Train Test Split<jupyter_code>from sklearn.model_selection import train_test_split
X = train.drop("Survived",axis=1)
y = train['Survived']
X_train,X_test,y_train,y_test= train_test_split(X, y,test_size=0.30, random_state=101)<jupyter_output><empty_output><jupyter_text>Training and Predicting<jupyter_code>from sklearn.linear_model import LogisticRegression
logmodel = LogisticRegression()
logmodel.fit(X_train,y_train)
predictions = logmodel.predict(X_test)<jupyter_output><empty_output><jupyter_text>Let's move on to evaluate our model!  EvaluationWe can check precision,recall,f1-score using classification report!<jupyter_code>from sklearn.metrics import classification_report,accuracy_score
print(classification_report(y_test,predictions))
accuracy_score(y_test, predictions)
from sklearn.metrics import  confusion_matrix
confusion_matrix(y_test,predictions)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07795282017794465, 'fraction_numerical': 0.006448453187494899, 'mean_word_length': 3.512707182320442, 'pattern_counts': {'":': 0, '<': 53, '<?xml version=': 0, '>': 53, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 31, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '340253', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 3354, 'n_tokens_neox': 3177, 'n_words': 1479, 'path': 'Logistic Regression/.ipynb_checkpoints/Logistic Regression with Python-Copy1-checkpoint.ipynb', 'repo_name': 'CommunityOfCoders/ML_Workshop_Teachers', 'size': '12251'}