<jupyter_start><jupyter_text>Build your first machine learning model<jupyter_code># Import modules
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import re
import numpy as np
from sklearn import tree
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV

# Figures inline and set visualization style
%matplotlib inline
sns.set()

# Import data
df_train = pd.read_csv('data/train.csv')
df_test = pd.read_csv('data/test.csv')<jupyter_output><empty_output><jupyter_text>* Below, you will drop the target 'Survived' from the training dataset and create a new DataFrame `data` that consists of training and test sets combined;* But first, you'll store the target variable of the training data for safe keeping.<jupyter_code># Store target variable of training data in a safe place
survived_train = df_train.Survived

# Concatenate training and test sets
data = pd.concat([df_train.drop(['Survived'],axis=1),df_test])<jupyter_output><empty_output><jupyter_text>* Check out your new DataFrame `data` using the `info()` method.<jupyter_code>data.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 1309 entries, 0 to 417
Data columns (total 11 columns):
PassengerId    1309 non-null int64
Pclass         1309 non-null int64
Name           1309 non-null object
Sex            1309 non-null object
Age            1046 non-null float64
SibSp          1309 non-null int64
Parch          1309 non-null int64
Ticket         1309 non-null object
Fare           1308 non-null float64
Cabin          295 non-null object
Embarked       1307 non-null object
dtypes: float64(2), int64(4), object(5)
memory usage: 122.7+ KB<jupyter_text>^ There are 2 numerical variables that have missing values: what are they?* Impute these missing values, using the median of the of these variables where we know them:<jupyter_code># Impute missing numerical variables
data['Age'] = data.Age.fillna(data.Age.median())
data['Fare'] = data.Fare.fillna(data.Fare.median())

# Check out info of data
data.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 1309 entries, 0 to 417
Data columns (total 11 columns):
PassengerId    1309 non-null int64
Pclass         1309 non-null int64
Name           1309 non-null object
Sex            1309 non-null object
Age            1309 non-null float64
SibSp          1309 non-null int64
Parch          1309 non-null int64
Ticket         1309 non-null object
Fare           1309 non-null float64
Cabin          295 non-null object
Embarked       1307 non-null object
dtypes: float64(2), int64(4), object(5)
memory usage: 122.7+ KB<jupyter_text>* As you want to encode your data with numbers, you'll want to change 'male' and 'female' to numbers. Use the `pandas` function `get_dummies` to do so:<jupyter_code>data = pd.get_dummies(data,columns=['Sex'],drop_first=True)
data.head()<jupyter_output><empty_output><jupyter_text>* Select the columns `['Sex_male', 'Fare', 'Age','Pclass', 'SibSp']` from your DataFrame to build your first machine learning model:<jupyter_code># Select columns and view head
data = data[['Sex_male', 'Fare', 'Age','Pclass', 'SibSp']]
data.head()<jupyter_output><empty_output><jupyter_text>* Use `.info()` to check out `data`:<jupyter_code>data.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 1309 entries, 0 to 417
Data columns (total 5 columns):
Sex_male    1309 non-null uint8
Fare        1309 non-null float64
Age         1309 non-null float64
Pclass      1309 non-null int64
SibSp       1309 non-null int64
dtypes: float64(2), int64(2), uint8(1)
memory usage: 52.4 KB<jupyter_text>**Recap:*** You've got your data in a form to build first machine learning model.**Up next:** it's time to build your first machine learning model!For more on `pandas`, check out our [Data Manipulation with Python track](https://www.datacamp.com/tracks/data-manipulation-with-python). If you're enoying this session, retweet or share on FB now and follow us on Twitter: [@hugobowne](https://twitter.com/hugobowne) & [@DataCamp](https://twitter.com/datacamp).  In which you build a decision tree classifier What is a Decision tree classsifier? It is a tree that allows you to classify data points (aka predict target variables) based on feature variables. For example,  * You first **fit** such a model to your training data, which means deciding (based on the training data) which decisions will split at each branching point in the tree: e.g., that the first branch is on 'Male' or not and that 'Male' results in a prediction of 'Dead'.  * Before fitting a model to your `data`, split it back into training and test sets:<jupyter_code>data_train = data.iloc[:891]
data_test = data.iloc[891:]<jupyter_output><empty_output><jupyter_text>* You'll use `scikit-learn`, which requires your data as arrays, not DataFrames so transform them:<jupyter_code>X = data_train.values
test = data_test.values
y = survived_train.values<jupyter_output><empty_output><jupyter_text>* Now you get to build your decision tree classifier! First create such a model with `max_depth=3` and then fit it your data:<jupyter_code># Instantiate model and fit to data
clf = tree.DecisionTreeClassifier(max_depth=3)
clf.fit(X,y)<jupyter_output><empty_output><jupyter_text>* Make predictions on your test set, create a new column 'Survived' and store your predictions in it. Save 'PassengerId' and 'Survived' columns of `df_test` to a .csv and submit to Kaggle.<jupyter_code># Make predictions and store in 'Survived' column of df_test
Y_pred = clf.predict(test)
df_test['Survived'] = Y_pred
df_test[['PassengerId', 'Survived']].to_csv('data/predictions/1st_dec_tree.csv', index=False)<jupyter_output><empty_output><jupyter_text>* What is the accuracy of your model, as reported by Kaggle?Accuracy = 77.9% **Recap:*** You've got your data in a form to build first machine learning model.* You've built your first machine learning model: a decision tree classifier.**Up next:** figure out what this `max_depth` argument was, why we chose it and explore `train_test_split`.For more on `scikit-learn`, check out our [Supervised Learning with scikit-learn course](https://www.datacamp.com/courses/supervised-learning-with-scikit-learn). If you're enoying this session, retweet or share on FB now and follow us on Twitter: [@hugobowne](https://twitter.com/hugobowne) & [@DataCamp](https://twitter.com/datacamp).  What was this decision tree classifier?  Note: you can use `graphviz` to generate figures such as this. See the `scikit-learn` documentation [here](http://scikit-learn.org/stable/modules/tree.html) for further details. In building this model, what you're essentially doing is creating a _decision boundary_ in the space of feature variables, for example (image from [here](http://scikit-learn.org/stable/auto_examples/ensemble/plot_voting_decision_regions.html)):   Why would you choose max_depth=3 ? The depth of the tree is known as a hyperparameter, which means a parameter we need to decide before we fit the model to the data. If we choose a larger `max_depth`, we'll get a more complex decision boundary. * If our decision boundary is _too complex_ we can overfit to the data, which means that our model will be describing noise as well as signal.* If our max_depth is too small, we may be underfitting the data, meaning that our model doesn't contain enough of the signal.**How do we tell whether we're overfitting or underfitting?** Note: this is also referred to as the bias-variance trade-off and we won;t go into details on that here. One way is to hold out a test set from our training data. We can then fit the model to our training data, make predictions on our test set and see how well our prediction does on the test set. * You'll now do this: split your original training data into training and test sets:<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.33, random_state=42, stratify=y)<jupyter_output><empty_output><jupyter_text>* Iterate over values of `max_depth` ranging from 1 to 9 and plot the accuracy of the models on training and test sets:<jupyter_code># Setup arrays to store train and test accuracies
dep = np.arange(1, 9)
train_accuracy = np.empty(len(dep))
test_accuracy = np.empty(len(dep))

# Loop over different values of k
for i, k in enumerate(dep):
    # Setup a k-NN Classifier with k neighbors: knn
    clf = tree.DecisionTreeClassifier(max_depth=k)

    # Fit the classifier to the training data
    clf.fit(X_train, y_train)
    
    #Compute accuracy on the training set
    train_accuracy[i] = clf.score(X_train, y_train)

    #Compute accuracy on the testing set
    test_accuracy[i] = clf.score(X_test, y_test)

# Generate plot
plt.title('clf: Varying depth of tree')
plt.plot(dep, test_accuracy, label = 'Testing Accuracy')
plt.plot(dep, train_accuracy, label = 'Training Accuracy')
plt.legend()
plt.xlabel('Depth of tree')
plt.ylabel('Accuracy')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09047724259832081, 'fraction_numerical': 0.025519222271321255, 'mean_word_length': 4.792066538707614, 'pattern_counts': {'":': 0, '<': 53, '<?xml version=': 0, '>': 53, 'https://': 6, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '53569', 'license': 'MIT', 'n_tokens_mistral': 2996, 'n_tokens_neox': 2778, 'n_words': 1067, 'path': '2-titanic_first_ML-model.ipynb', 'repo_name': 'dcadenasa/titanic', 'size': '9052'}