<jupyter_start><jupyter_text>Titanic Dataset InvestigationThe sinking of the RMS Titanic is one of the most infamous shipwrecks in history.  On April 15, 1912, during her maiden voyage, the Titanic sank after colliding with an iceberg, killing 1502 out of 2224 passengers and crew.One of the reasons that the shipwreck led to such loss of life was that there were not enough lifeboats for the passengers and crew. Although there was some element of luck involved in surviving the sinking, some groups of people were more likely to survive than others, such as women, children, and the upper-class.This notebook describes the data analysis process that I took as I analyzed data from 891 of the 2224 passengers and crew on board of the Titanic and looked at what factors made people more likely to survive the tragedy.The dataset used was provided by [Kaggle](https://www.kaggle.com).  AcquiringFirst, I need to import all the libraries and packages that are necessary for this notebook and create a pandas dataframe from the titanic-data.csv file.<jupyter_code># import all necessary packages and functions
import csv
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

%matplotlib inline
# file location
dataset = "titanic-data.csv"

# creating a pandas dataframe
titanic_data = pd.read_csv(dataset)<jupyter_output><empty_output><jupyter_text>CleaningNow, we have some data but after looking at some of the values carefully I noticed that some of them were missing. Moreover, not all this information is relevant for my analysis. For example, I don't really need the passenger's ID number or their ticket number.On the other side, a part of the data is not represented well. It's better to group age together by ranges than to have individual values for each passenger. Also, I'm only concerned with whether a passenger has a family member aboard or not. It's not important for my analysis if this family member is a sibling or a parent.<jupyter_code># new column which represents the prescence of family aboard or not
titanic_data['Family'] = (titanic_data['SibSp'] == 1) | (titanic_data['Parch'] == 1)

age_labels = ['0-9', '10-19', '20-29', '30-39', '40-49', '50-59', '60-69', '70-79']

# new column which makes age more easy to analyze by grouping it together in ranges
titanic_data['Age_group'] = pd.cut(titanic_data.Age, range(0, 81, 10), right=False, labels=age_labels)

# removing data that will not be used
titanic_data.drop(['PassengerId', 'SibSp', 'Parch', 'Ticket', 'Cabin', 'Age'], axis=1, inplace=True)<jupyter_output><empty_output><jupyter_text>Still, I'm not satisfied by how the dataset looks. Some of the columns can have better descriptions and some of the values can be better represented using booleans.<jupyter_code># improving the value types in the 'survived' column
survived_map = {0: False, 1: True}
titanic_data['Survived'] = titanic_data['Survived'].map(survived_map)

# improving the descriptions in the 'Class' column
class_map = {1: 'Upper Class', 2: 'Middle Class', 3: 'Lower Class'}
titanic_data['Class'] = titanic_data['Pclass'].map(class_map)

# improving the descriptions in the 'Port' column
port_map = {'S': 'Southampton', 'C': 'Cherbourg','Q':'Queenstown'}
titanic_data['Port'] = titanic_data['Embarked'].map(port_map)

# removing the Embarked column
titanic_data.drop(['Embarked', 'Pclass'], axis=1, inplace=True)

# displaying the data
titanic_data.head()
titanic_data.isnull().sum()<jupyter_output><empty_output><jupyter_text>The dataset is not complete though. Some rows have missing information from the age group and port columns. To handle this, I will create a new dataframe that excludes those entries. I won't use this new dataframe when the age group or port columns are not relevant since I don't want to reduce my sample size when data from name, sex, fare, family, or class is still present.<jupyter_code># creates a new dataframe with no missing values
titanic_data_full = titanic_data.dropna(axis=0)<jupyter_output><empty_output><jupyter_text>ExploringNow that the dataset looks much better I can start exploring and learning from it. I'm concerned with the factors that made people more likely to survive the sinking of the Titanic. For now, I'll find the number of survivors from the 891 passengers and the general probability of surviving.<jupyter_code># number of passengers in the dataset
number_of_passengers = len(titanic_data)

# number_of_survivors
number_of_survivors = (titanic_data['Survived']).sum()

# survival probability
survival_probability = number_of_survivors / number_of_passengers

print("Total number of passengers: " + str(number_of_passengers))
print("Total survivors: " + str(number_of_survivors))
print("Probability of survival: " + str(round(survival_probability, 2)))
plot = sns.countplot(data=titanic_data, x="Survived", palette="Blues_d")
plot.set_title("Non-survivors vs survivors")
plt.show()<jupyter_output><empty_output><jupyter_text>Survival and genderWhen the Titanic sank, the safety of women and children came first. My first guess is that their is a higher ratio of women how survived than men.<jupyter_code># number of men on board of the Titanic
number_of_men = (titanic_data['Sex'] == 'male').sum()

# number of women on board of the Titanic
number_of_women = (titanic_data['Sex'] == 'female').sum()

print("Number of men: " + str(number_of_men))
print("Number of women: " + str(number_of_women))
plot = sns.countplot(data=titanic_data, x="Sex", palette="Blues_d")
plot.set_title("Gender count")
plt.show()
# group the data by gender
gender_vs_survival = titanic_data.groupby(['Survived', 'Sex']).size()

# number of men who survived and did not survived
men_survived = gender_vs_survival[True, 'male']
men_not_survived = gender_vs_survival[False, 'male']

# number of women who survived and did not survived
women_survived = gender_vs_survival[True, 'female']
women_not_survived = gender_vs_survival[False, 'female']

# survival percentages
survival_probability_for_men = round(men_survived / number_of_men, 3)
survival_probability_for_women = round(women_survived / number_of_women, 3)

print("Number of men who survived: " + str(men_survived))
print("Number of women who survived: " + str(women_survived) + "\n")
print("Probability of surviving as a men: " + str(survival_probability_for_men))
print("Probability of surviving as a woman: " + str(survival_probability_for_women))
plot = sns.barplot(data=titanic_data, x="Sex", y="Survived", palette="Blues_d", estimator= lambda x: sum(x) / len(x))
plot.set_title("Probrability of survival by gender")
plot.set(ylabel="Probability")
plt.show()<jupyter_output><empty_output><jupyter_text>Women were much more likely to survive the tragedy than men as 74% of the woman abort survived while only a mere 19% of the men did.  Survival and ageBased on my previous guess, I also expect to see I higher rate of children who survived. Keep in mind that since the dataset has missing entries for the age column, I will be using the `titanic_data_full dataframe` for this part. This dataframe contains only those rows where the age and port values are present.<jupyter_code># Distribution of age by groups
distribution_of_ages = sns.countplot(x="Age_group", data=titanic_data_full, palette="Blues_d")
distribution_of_ages.set_title("Distribution of ages")
distribution_of_ages.set(xlabel="Age group")
plt.show(distribution_of_ages)<jupyter_output><empty_output><jupyter_text>As we can see from the bar plot, the shape of the age distribution is normal and most of the passengers were between 10 and 49 years old. I'm going to count as a children anyone younger than 9 years old.<jupyter_code>age_vs_survival = titanic_data_full.groupby(['Survived', 'Age_group']).size()

# probability_of_survival_by_age
probability_of_survival_by_age = sns.barplot(x="Age_group", y="Survived", data=titanic_data_full, palette="Blues_d", estimator=lambda x: sum(x) / len(x))
probability_of_survival_by_age.set_title("Probability of survival by age")
probability_of_survival_by_age.set(xlabel="Age group")
probability_of_survival_by_age.set(ylabel="Probability")
plt.show(probability_of_survival_by_age)<jupyter_output><empty_output><jupyter_text>The probability plot above reveals that children of ages 0-9 were the most likely to survive the tragedy. A saddening insight also reveals that none of the 6 passengers of ages 70-79 survived. Apart from the edge cases of being a child or an older passenger, the probability of survival doesn't seem to change much. It's almost as likely for someone of age 10-19 to survive as it is for someone of age 50-59.  Survival and social classIn the 1997 movie by <NAME>, we meet the fictional character <NAME> who is played by <NAME>. Hockley is the son of a successful steel tycoon and is able to pay for the whole trip as first class passengers due to his high wealth and social position. When the Titanic is sinking, he tries to pay his way into a lifeboat but the emergency personnel stops him and tells him that his money can't save him more that it could save the others aboard.I find this an interesting example since it means that the crew was following the "women and children only" rule and that no matter your wealth or social status you had the same chance of survival as anyone else. At least this was the case in the movie. Let's find out if it holds using real data.<jupyter_code># Distribution of social classes
distribution_of_class = sns.countplot(x="Class", data=titanic_data, palette="Blues_d")
distribution_of_class.set_title("Distribution of social classes")
plt.show(distribution_of_class)
class_vs_survival = titanic_data.groupby(['Survived', 'Class']).size()

# probability of survival by class
probability_of_survival_by_class = sns.barplot(x="Class", y="Survived", data=titanic_data, palette="Blues_d", estimator=lambda x: sum(x) / len(x))
probability_of_survival_by_class.set_title("Probability of survival by class")
probability_of_survival_by_class.set(xlabel="Class")
probability_of_survival_by_class.set(ylabel="Probability")
plt.show(probability_of_survival_by_class)<jupyter_output><empty_output><jupyter_text>Even though Hockley wasn't able to pay his way into the lifeboat (although he got in the boat later anyways), it appears that social class was a big factor affecting the likehood of survival. Uperclass passengers had a survival chance of 63% while lower class passenger only had a chance of 24%.  Survival and family presenceA final factor to consider is the presence of family member aboard or not. This can affect the chances of survival in many ways. It could be that a person doesn't wish to leave a loved one that can't make it out themself or family members helping each others as a group and overcoming obstacles that couldn't be done alone.<jupyter_code>plot = sns.barplot(data=titanic_data, x="Family", y="Survived", palette="Blues_d", estimator= lambda x: sum(x) / len(x))
plot.set_title("Probrability of survival by family presence")
plot.set(ylabel="Probability")
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08234656060743017, 'fraction_numerical': 0.009400705052878966, 'mean_word_length': 6.110539845758355, 'pattern_counts': {'":': 0, '<': 44, '<?xml version=': 0, '>': 44, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '640336', 'license': 'MIT', 'n_tokens_mistral': 3472, 'n_tokens_neox': 3232, 'n_words': 1320, 'path': 'Titanic Dataset Investigation.ipynb', 'repo_name': 'samyanez94/Titanic-Dataset-Investigation', 'size': '11083'}