<jupyter_start><jupyter_text>Titanic_Analysis_Basic Check out the Data Set at the following link:https://www.kaggle.com/c/titanic-gettingStartedDownload the train.csv file and save it to the same location as your iPython Notebooks.<jupyter_code># Import the packages
import pandas as pd
from pandas import Series,DataFrame
# Set up the Titanic csv file as a DataFrame
titanic_df = pd.read_csv('train.csv')

# General review of the dataset
titanic_df.head()
# Check data_type and size
titanic_df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 891 entries, 0 to 890
Data columns (total 12 columns):
PassengerId    891 non-null int64
Survived       891 non-null int64
Pclass         891 non-null int64
Name           891 non-null object
Sex            891 non-null object
Age            714 non-null float64
SibSp          891 non-null int64
Parch          891 non-null int64
Ticket         891 non-null object
Fare           891 non-null float64
Cabin          204 non-null object
Embarked       889 non-null object
dtypes: float64(2), int64(5), object(5)
memory usage: 83.6+ KB<jupyter_text>Some basic questions have to know fist:1. Who were the passengers on the Titanic? (Ages,Gender,Class,..etc)2. What deck were the passengers on and how does that relate to their class?3. Where did the passengers come from?4. Who was alone and who was with family?Maybe think more: - What factors helped someone survive the sinking?<jupyter_code># Import packages
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline<jupyter_output><empty_output><jupyter_text>First let's look at gender:<jupyter_code>sns.catplot('Sex',data=titanic_df,kind="count")
# You can also change your preference of color by passing the argument【 palette='winter'】or 【 palette='summer'】
# eample ： sns.catplot('Sex',data=titanic_df,kind="count",palette='winter')
# We can seperate the genders by classes, using the 'hue' arguement !
sns.catplot('Pclass',data=titanic_df,kind="count",hue='Sex')<jupyter_output><empty_output><jupyter_text>Well, it seems that there are more males and child in the 3rd class than females. Also,it might be useful to know the split between males,females,and children. We treat anyone as under 16 as a child, here we use the apply function to create a new column named ['person'] based on Sex, Age<jupyter_code># There are also other more simple ways to realize it
def male_female_child(passenger):
    # Take the Age and Sex
    age,sex = passenger
    # Compare the age, otherwise leave the sex
    if age < 16:
        return 'child'
    else:
        return sex
    

# We'll define a new column called 'person', add axis=1 for columns and not index
titanic_df['person'] = titanic_df[['Age','Sex']].apply(male_female_child,axis=1)
# Let's see if this worked, check out the first ten rows
titanic_df[0:10]<jupyter_output><empty_output><jupyter_text>Success ! Now we have seperated the passengers between female,male,and child.Now it's time to prove  the famous "Women and children first policy"  ^.^<jupyter_code># Using the carplot again!
#sns.catplot('Pclass',data=titanic_df,kind='count',hue='person')
sns.catplot(y="Pclass", hue="person", kind="count",
            palette="pastel", edgecolor=".6",
            data=titanic_df)<jupyter_output><empty_output><jupyter_text>It's much better to see the bigger picture via swarm graph<jupyter_code>sns.catplot(x='Pclass', y='Age',hue="person",aspect=.90,kind="swarm", data=titanic_df)<jupyter_output><empty_output><jupyter_text>There are more children in the 3rd class, not so many in the 1st class, we need a distribution here to get a more precise picture<jupyter_code># A histogram of Age with pandas
titanic_df['Age'].hist(bins=70)
# A quick overall comparison of male,female,child
titanic_df['person'].value_counts()
# Another way to visualize the data is to use FacetGrid to plot multiple kedplots on one plot

# Set the figure equal to a facetgrid with the pandas dataframe as its data source, set the hue, and change the aspect ratio.
fig = sns.FacetGrid(titanic_df, hue="Sex",aspect=4)

# Next use map to plot all the possible kdeplots for the 'Age' column by the hue choice
fig.map(sns.kdeplot,'Age',shade= True)

# Set the x max limit by the oldest passenger
oldest = titanic_df['Age'].max()

#Since we know no one can be negative years old set the x lower limit at 0
fig.set(xlim=(0,oldest))

#Finally add a legend
fig.add_legend()<jupyter_output><empty_output><jupyter_text>Here we get the distribution of age by sex, we'll do the same thing for column "person" and "Pclass"<jupyter_code># We could have done the same thing for the 'person' column to include children:

fig = sns.FacetGrid(titanic_df, hue="person",aspect=4)
fig.map(sns.kdeplot,'Age',shade= True)
oldest = titanic_df['Age'].max()
fig.set(xlim=(0,oldest))
fig.add_legend()
# Let's do the same for class by changing the hue argument
fig = sns.FacetGrid(titanic_df, hue="Pclass",aspect=4)
fig.map(sns.kdeplot,'Age',shade= True)
oldest = titanic_df['Age'].max()
fig.set(xlim=(0,oldest))
fig.add_legend()<jupyter_output><empty_output><jupyter_text>So far, we have answered the first question, we know who are the passengers based on Sex，Age，Class, we can continue to the second question: What deck were the passengers on and how does that relate to their class?<jupyter_code># Quick review of our dataset
titanic_df.head()
# Drop the Null values of the column "Cabin", name it "deck"
deck = titanic_df['Cabin'].dropna()
# Quick preview of the decks
deck.head()
# We can find out that there are 687 missing values for the column "Cabin"
titanic_df.shape[0]-deck.shape[0]<jupyter_output><empty_output><jupyter_text>We only need the first letter of the deck to classify its level (e.g. A,B,C,D,E,F,G)<jupyter_code>#let's grab that letter for the deck level with a simple for loop

# Set empty list
levels = []

# Loop to grab first letter
for level in deck:
    levels.append(level[0])    

# Reset DataFrame and use factor plot
cabin_df = DataFrame(levels)
cabin_df.columns = ['Cabin']
sns.catplot('Cabin',data=cabin_df,kind="count",palette="ch:.25")<jupyter_output><empty_output><jupyter_text>We can notive that there is  a 'T' deck value there which doesn't make sense, we can drop it out<jupyter_code># Redefine cabin_df  where the row doesn't equal to 'T'
cabin_df = cabin_df[cabin_df.Cabin != 'T']
#Replot
#sns.catplot('Cabin',data=cabin_df,kind="count",palette='summer')
f, ax = plt.subplots(figsize=(7, 3))
sns.countplot(y="Cabin", data=cabin_df, color="c");<jupyter_output><empty_output><jupyter_text>Note that we can use 'summer_d' or 'winter_d' to make the color darker,now that we've analyzed the distribution by decks, more survivors from deck B and C, now we continue to the third question:3.) Where did the passengers come from?<jupyter_code>titanic_df.Embarked.dropna().value_counts()<jupyter_output><empty_output><jupyter_text>If we look into the Embarked column in the dataset, we can see only there are only three values : S,C,Q, which stands for Cherbourg, Queenstown, Southhampton<jupyter_code>#Now we can make a quick factorplot to check out the results, note the x_order argument, used to deal with NaN values

sns.catplot('Embarked',data=titanic_df,hue='Pclass',kind="count",order=['C','Q','S'])<jupyter_output><empty_output><jupyter_text>Compared with other 2 places, we can notice that almost all the passagers from Queenstown that boarded 3rd class,In order to find out the raison behind it, maybe we can search some economic data, distribution of social class... of Queenstown in that time Now let's take a look at the last question:4.) Who was alone and who was with family?<jupyter_code># First of all, we can add a new column to define alone
# We'll add the parent/child column with the sibsp column to calculate the total number of one's relatives
titanic_df['Alone'] =  titanic_df.Parch + titanic_df.SibSp
titanic_df['Alone'].head()
# Look for >0 or ==0 to set alone status
titanic_df['Alone'].loc[titanic_df['Alone'] >0] = 'With Family'
titanic_df['Alone'].loc[titanic_df['Alone'] == 0] = 'Alone'
# Check the dataframe after the modification
titanic_df.head()
# Now let's get a simple visualization!
# There so many color palettes to choose, see more : "https://seaborn.pydata.org/tutorial/color_palettes.html"
sns.catplot('Alone',data=titanic_df,kind="count",palette='colorblind')<jupyter_output><empty_output><jupyter_text>So far we already got some basic information from the dataset, now it's time to think a little more:What factors helped someone survive the sinking?Since we know that in the column "Survived",0 stands for No,1 stands for yes,we can use map to change this column<jupyter_code>titanic_df["Survivor"] = titanic_df.Survived.map({0: "no", 1: "yes"})
sns.catplot('Survivor',data=titanic_df,kind="count",palette='summer')<jupyter_output><empty_output><jupyter_text>It seems that more people died than those who survived. Let's see if the class has an effect on their survival rate<jupyter_code>sns.catplot('Pclass','Survived',kind="point",data=titanic_df)
g = sns.catplot(x="person", y="Survived", col="Pclass",
                 data=titanic_df, saturation=.5,
                kind="bar", ci=None, aspect=.6)
(g.set_axis_labels("", "Survival Rate")
.set_xticklabels(["Men", "Women", "Children"])
.set_titles("{col_name} {col_var}")
.set(ylim=(0, 1))
.despine(left=True))<jupyter_output><empty_output><jupyter_text>It looks like survival rates for the 3rd class are much more lower than 1st and 2nd.But maybe this effect is caused by the large amount of men in the 3rd class in combination with the women and children first policy. Let's use 'hue' to get a clearer picture on this.<jupyter_code>sns.catplot('Pclass','Survived',hue='person',kind="point",data=titanic_df)<jupyter_output><empty_output><jupyter_text>From the result we know either being a male or being in 3rd class has lower rate of survival. But what about age? Did being younger or older have an effect on survival rate?<jupyter_code># Let's use a linear plot on age versus survival
sns.lmplot('Age','Survived',data=titanic_df)<jupyter_output><empty_output><jupyter_text>Maybe it's more clear with a Hexbin plot!<jupyter_code>sns.jointplot('Age','Survived',data=titanic_df, kind="hex", color="#4CB391")<jupyter_output><empty_output><jupyter_text>General trend is that the older the passenger was, the less likely they survived. Also we can find out that the average age range is between 20 and 40Now we use hue to take a look at the effect of class and age.<jupyter_code>sns.lmplot('Age','Survived',hue='Pclass',data=titanic_df,palette='Set2')<jupyter_output><empty_output><jupyter_text>We can also use the x_bin argument to clean up this figure and grab the data and bin it by age with a std attached!<jupyter_code>generations=[0,10,20,40,60,80]
sns.lmplot('Age','Survived',hue='Pclass',data=titanic_df,palette='Set2',x_bins=generations)<jupyter_output><empty_output><jupyter_text>Inseresting to find older 1st class passengers, higher survival rate ! What about if we relate gender and age with the survival set?<jupyter_code>sns.lmplot('Age','Survived',hue='Sex',data=titanic_df,palette='Set2',x_bins=generations)
sns.jointplot('Age','Survived',data=titanic_df, kind="hex", color="#4CB391")
Just like what we thought, women and children first, some brilliance of human nature here !<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10525854513584575, 'fraction_numerical': 0.013584574934268186, 'mean_word_length': 5.780154486036839, 'pattern_counts': {'":': 0, '<': 90, '<?xml version=': 0, '>': 91, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '22', 'dir': 'jupyter-structured-clean-dedup', 'id': '659034', 'license': 'MIT', 'n_tokens_mistral': 3957, 'n_tokens_neox': 3726, 'n_words': 1264, 'path': 'Titanic_Analysis_Basic.ipynb', 'repo_name': 'yaozeliang/Data_Analysis_Titanic', 'size': '11410'}