<jupyter_start><jupyter_text>Pipeline: Split data into train, validation, and test setUsing the Titanic dataset from [this](https://www.kaggle.com/c/titanic/overview) Kaggle competition.In this section, we will split the data into train, validation, and test set in preparation for fitting a basic model in the next section.  Read in Data<jupyter_code>import pandas as pd
from sklearn.model_selection import train_test_split

titanic_df = pd.read_csv('../Data/titanic_cleaned.csv')
titanic_df.head()<jupyter_output><empty_output><jupyter_text>Split into train, validation, and test set![Split Data](../img/split_data.png)<jupyter_code># features columns and label column
features = titanic_df.drop(['Survived'], axis=1)
labels = titanic_df['Survived']
# split train data 60%, validation data 20% , test data 20%
X_train, X_test, y_train, y_test = train_test_split(features, labels, test_size=0.4, random_state=42)
# split again 50%, 50% on test data set for validation, test
X_test, X_val, y_test, y_val = train_test_split(X_test, y_test, test_size=0.5, random_state=42)
# quick check on the data numbers
for dataset in [y_train, y_test, y_val]:
    print(round(len(dataset) / len(labels), 2))<jupyter_output>0.6
0.2
0.2<jupyter_text>Write out data<jupyter_code>X_train.to_csv('../Data/train_features.csv', index=False)
X_val.to_csv('../Data/val_features.csv', index=False)
X_test.to_csv('../Data/test_features.csv', index=False)

y_train.to_csv('../Data/train_labels.csv', index=False)
y_val.to_csv('../Data/val_labels.csv', index=False)
y_test.to_csv('../Data/test_labels.csv', index=False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1313193588162762, 'fraction_numerical': 0.016029593094944512, 'mean_word_length': 6.410958904109589, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '95429', 'license': 'MIT', 'n_tokens_mistral': 600, 'n_tokens_neox': 578, 'n_words': 137, 'path': 'ML - Applied Machine Learning Foundation/05.End to End Pipeline/03.Pipeline - Split data into train, validation, and test set.ipynb', 'repo_name': 'ptyadana/probability-and-statistics-for-business-and-data-science', 'size': '1622'}