<jupyter_start><jupyter_text>Titanic Survival Prediction   Data DictionaryHere we got the data dictionary. This useful info will help you to understand the csv.|Variable	|Definition	|Key||:----|:-----|:----||survival|	Survival|	0 = No, 1 = Yes||pclass|	Ticket class|	1 = 1st, 2 = 2nd, 3 = 3rd||sex|	Sex	| ||Age|	Age in years| |	|sibsp|	 of siblings / spouses aboard the Titanic| ||parch|	 of parents / children aboard the Titanic|	 ||ticket|	Ticket number|	 ||fare|	Passenger fare|	 ||cabin|	Cabin number| |	|embarked|	Port of Embarkation|	C = Cherbourg, Q = Queenstown, S = Southampton | Variable Notes**pclass:** A proxy for socio-economic status (SES)1st = Upper2nd = Middle3rd = Lower**age:** Age is fractional if less than 1. If the age is estimated, is it in the form of xx.5**sibsp:** The dataset defines family relations in this way...Sibling = brother, sister, stepbrother, stepsisterSpouse = husband, wife (mistresses and fiancés were ignored)**parch:** The dataset defines family relations in this way...Parent = mother, fatherChild = daughter, son, stepdaughter, stepsonSome children travelled only with a nanny, therefore parch=0 for them.Please check https://www.kaggle.com/c/titanic/data **Let's waggle the data!!!**  Import useful modules<jupyter_code>%matplotlib inline

import matplotlib.pyplot as plt
import pandas as pd

plt.style.use('ggplot')

# Anaconda on Windows will get warning<jupyter_output><empty_output><jupyter_text>Loading data from CSV<jupyter_code>df=pd.read_csv('titanic.csv')<jupyter_output><empty_output><jupyter_text>Show first 5 records<jupyter_code>df.head()<jupyter_output><empty_output><jupyter_text>HistogramNice tool to help you understand the data<jupyter_code>df.hist( figsize=(16, 10))<jupyter_output><empty_output><jupyter_text>Show statistic<jupyter_code>df.describe()<jupyter_output><empty_output><jupyter_text>Show list of column names<jupyter_code>df.keys()<jupyter_output><empty_output><jupyter_text>Drop some unnecessary columns<jupyter_code>df2=df.drop(labels=['Name','PassengerId','Cabin','Ticket'],axis=1)
df2.head()<jupyter_output><empty_output><jupyter_text>From column 'Sex', create another numeric column (sexc)Randonforest cannot process text data. All data must be numeric type.<jupyter_code>code={'female':0,'male':1}
for k,i in df2.iterrows():
    df2.loc[k,"sexc"]=code[i['Sex']]
df2.head()<jupyter_output><empty_output><jupyter_text>Count number of records of each category<jupyter_code>df2["Embarked"].value_counts()<jupyter_output><empty_output><jupyter_text>From column 'Embarked', create another numeric column (embarkedc)<jupyter_code>code={'C':0,'Q':1, 'S':2}
for k,i in df2.iterrows():
    if i['Embarked']==i['Embarked']:
        df2.loc[k,"embarkedc"]=code[i['Embarked']]
df2.head(10)
#remove the original column
df3=df2.drop(['Sex','Embarked'],1)<jupyter_output><empty_output><jupyter_text>Drop every NaN row<jupyter_code>df4=df3.dropna(axis=0)
df4.head(10)<jupyter_output><empty_output><jupyter_text>Alternative approuch--drop every NaN row using Masking<jupyter_code>df5=df3[ df3['Age'].notnull() ]
df5.head()<jupyter_output><empty_output><jupyter_text>Convert Dataframe to numpy array for traning<jupyter_code>df4.head()
X=df4[df4.columns[1:]].values
x_names=list(df4.columns[1:])
y=df4.Survived.values
y_names=['No','Yes']
y.shape
y=y.reshape(-1)
y.shape
X.shape
X<jupyter_output><empty_output><jupyter_text>Construct Classifier<jupyter_code>from sklearn.ensemble import ExtraTreesClassifier,RandomForestClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import cross_val_score
forest = ExtraTreesClassifier(n_estimators=250,random_state=0)<jupyter_output><empty_output><jupyter_text>Lazy k-fold cross-valivation<jupyter_code>scores = cross_val_score(forest, X, y, cv=5)
scores.mean()<jupyter_output><empty_output><jupyter_text>Training<jupyter_code>forest.fit(X,y)<jupyter_output><empty_output><jupyter_text>Training<jupyter_code>from sklearn.tree import export_graphviz
#forest = RandomForestClassifier(n_estimators=10,random_state=0, max_depth=3)
#forest.fit(X,y)
#estimator = forest.estimators_[0]

estimator=DecisionTreeClassifier(max_depth=2)
estimator.fit(X,y)
# Export as dot file
export_graphviz(estimator, out_file='tree.dot', 
                feature_names =  x_names,
                class_names = y_names,
                rounded = True, proportion = True, 
                precision = 1, filled = True)

# Convert to png using system command (requires Graphviz)
from subprocess import call
call(['dot', '-Tpng', 'tree.dot', '-o', 'tree.png', '-Gdpi=600'])

# Display in jupyter notebook
from IPython.display import Image
Image(filename = 'tree.png')<jupyter_output><empty_output><jupyter_text>Prediction<jupyter_code>#Pclass=3 (lower), Age=22 , SibSp=1, Parch=0, Fare=7.25 (free trip)
#sexc=1 (male), embarkedc=1 (Queenstown)
forest.predict([[3,22.0,1,0,7.25,1.0,1.0]])
#Pclass=3 (lower), Age=22 , SibSp=1, Parch=0, Fare=7.25 (free trip)
#sexc=0 (male), embarkedc=1 (Queenstown)
forest.predict([[3,22.0,1,0,7.25,0.0,1.0]])<jupyter_output><empty_output><jupyter_text>Dummy Jack<jupyter_code>#Pclass=3 (lower), Age=25 (guessing), SibSp=0, Parch=0, Fare=0.0 (free trip), sexc=1 (male), embarkedc=1 (Queenstown, guessing)
forest.predict([[3,25.0,0,0,0.0,1,1]])<jupyter_output><empty_output><jupyter_text>*--The output of prediction is 0 that means sorry for Jack, he's not survive.*  Dummy Rose<jupyter_code>forest.predict([[1,23.0,0,1,50.0,0,1]])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14347984017435525, 'fraction_numerical': 0.027061387577188522, 'mean_word_length': 7.407633587786259, 'pattern_counts': {'":': 0, '<': 81, '<?xml version=': 0, '>': 81, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '20', 'dir': 'jupyter-structured-clean-dedup', 'id': '559585', 'license': 'BSD-2-Clause', 'n_tokens_mistral': 2168, 'n_tokens_neox': 2048, 'n_words': 406, 'path': '01day03_etl1/notebook_example/01 import CSV.ipynb', 'repo_name': 'wasit7/DataScience', 'size': '5506'}