<jupyter_start><jupyter_text>Exploratory data analysis of the UCI Bank Marketing data setThis is the exploratory data analysis for our data analysis [proposal](https://github.com/UBC-MDS/DSCI_522_Group_10).<jupyter_code>import warnings
warnings.filterwarnings('ignore')


import numpy as np
import pandas as pd

bank_df = pd.read_csv("../data/raw/bank-additional-full.csv", sep=";")
train_df = pd.read_csv("../data/processed/bank-additional-full_train.csv")
test_df = pd.read_csv("../data/processed/bank-additional-full_test.csv")<jupyter_output><empty_output><jupyter_text>Summary of the data set The data we are using for this project, [bank-additional-full.csv](https://archive.ics.uci.edu/ml/machine-learning-databases/00222/bank-additional.zip), was from a marketing campaign of a Portuguese bank. It was sourced from the UCI Learning Repository and can be found on this [website](https://archive.ics.uci.edu/ml/datasets/Bank+Marketing).Each row of the data is related to the direct telemarketing campaigns. They were using telemarketing to attempt to get customer to sign up for the bank's term deposit product.  The target in this dataset is yes or no to subscribing to the term deposit product.There are also some values of 'unknown' in some categorical features like education. We are considering imputation but will re-assess this while preprocessing the features.<jupyter_code>total_examples = len(bank_df)
total_features = len(bank_df.columns) - 1
count_NA = bank_df.isna().sum().sum()
print(f"There is a total of {total_examples} examples, {total_features} features and {str(count_NA)} observations with missing values in the dataset")
class_distribution = bank_df.y.value_counts().to_frame().T.rename(columns={"no":"Not Purchased", "yes":"Purchased"}, index={"y":"Class Distribution"})
class_distribution<jupyter_output><empty_output><jupyter_text>Table 1. Counts of observation for each class.The data used in this analysis is very simliar to the data used in [Moro et al., 2014].  Partition the data set into training and test sets Before proceeding further, we will split the data such that 80% of observations are in the training and 20% of observations are in the test set. Below we list the counts of observations for each class:<jupyter_code>train_class_distribution = train_df.target.value_counts().to_frame().T.rename(columns={0:"Not Purchased", 1:"Purchased"}, index={"target":"Training Set Class Distribution"})
test_class_distribution = test_df.target.value_counts().to_frame().T.rename(columns={0:"Not Purchased", 1:"Purchased"}, index={"target":"Test Set Class Distribution"})
pd.concat([train_class_distribution, test_class_distribution])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10530218761586949, 'fraction_numerical': 0.008898776418242492, 'mean_word_length': 6.331521739130435, 'pattern_counts': {'":': 5, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '420321', 'license': 'MIT', 'n_tokens_mistral': 823, 'n_tokens_neox': 772, 'n_words': 260, 'path': 'src/bank_marketing_data_eda.ipynb', 'repo_name': 'ChuckHo777/DSCI_522_Group_10', 'size': '2697'}