<jupyter_start><jupyter_text>Bank Marketing Data Analysis  Data Set Information: The data is related with direct marketing campaigns of a Portuguese banking institution. The marketing campaigns were based on phone calls. Often, more than one contact to the same client was required, in order to access if the product (bank term deposit) would be ('yes') or not ('no') subscribed.There are four datasets:1) bank-additional-full.csv with all examples (41188) and 21 inputs, ordered by date (from May 2008 to November 2010), very close to the data analyzed in [Moro et al., 2014]2) bank-additional.csv with 10% of the examples (4119), randomly selected from 1), and 21 inputs.3) bank-full.csv with all examples and 17 inputs, ordered by date (older version of this dataset with less inputs).4) bank.csv with 10% of the examples and 17 inputs, randomly selected from 3 (older version of this dataset with less inputs).The smallest datasets are provided to test more computationally demanding machine learning algorithms (e.g., SVM).The classification goal is to predict if the client will subscribe (yes/no) a term deposit (variable y)  Importing Libraries<jupyter_code>#import libraries
import seaborn as sns
import numpy as np
import scipy.stats as stats
import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd
from sklearn import preprocessing
from sklearn import model_selection
from sklearn import metrics
from sklearn.preprocessing import LabelEncoder
%matplotlib inline
plt.style.use('ggplot')
# loading csv data to pandas dataframe
bank_data = pd.read_csv( "../data/bank-additional-full.csv", sep=";")
print(bank_data.shape)
bank_data.head(5)<jupyter_output>(41188, 21)<jupyter_text>Checking Columns<jupyter_code>#list all columns
bank_data.columns<jupyter_output><empty_output><jupyter_text>Explanatory Data Analysis  Conducting EDA on data<jupyter_code>def examine_data_frame( df):
    for name in df.columns:
        print("----------")
        print(df[ name].dtype)
        if df[name].dtype is np.dtype( 'O'):
            print(df[name].value_counts())
            print("Name: ", name)
        else:
            print(df[ name].describe())

examine_data_frame(bank_data)<jupyter_output>----------
int64
count    41188.00000
mean        40.02406
std         10.42125
min         17.00000
25%         32.00000
50%         38.00000
75%         47.00000
max         98.00000
Name: age, dtype: float64
----------
object
admin.           10422
blue-collar       9254
technician        6743
services          3969
management        2924
retired           1720
entrepreneur      1456
self-employed     1421
housemaid         1060
unemployed        1014
student            875
unknown            330
Name: job, dtype: int64
Name:  job
----------
object
married     24928
single      11568
divorced     4612
unknown        80
Name: marital, dtype: int64
Name:  marital
----------
object
university.degree      12168
high.school             9515
basic.9y                6045
professional.course     5243
basic.4y                4176
basic.6y                2292
unknown                 1731
illiterate                18
Name: education, dtype: int64
Name:  education
----------
object
no         3[...]<jupyter_text>Checking for missing values<jupyter_code># Check missing values
bank_data.isna().sum()<jupyter_output><empty_output><jupyter_text>There is no missing values in columns<jupyter_code>categorical_columns = ['job','marital','education','default','housing','loan','contact','month','day_of_week',
                      'poutcome']
numerical_columns = ['age','duration','campaign','pdays','previous','emp.var.rate','cons.price.idx',
                     'cons.conf.idx','euribor3m','nr.employed']
bank_data[numerical_columns].describe()<jupyter_output><empty_output><jupyter_text>Plot Univariate analysis<jupyter_code>#plot histogram for all numerical variables
bank_data[numerical_columns].hist(bins=20, figsize=(15, 10))
plt.show()
#plot numerical variable in relation to term deposit subscription
fig, ax = plt.subplots(3, 3, figsize=(15, 10))
bank_data[bank_data.y == "no"][numerical_columns[:-1]].hist(bins=20, color="blue", alpha=0.5, ax=ax)
bank_data[bank_data.y == "yes"][numerical_columns[:-1]].hist(bins=20, color="red", alpha=0.5, ax=ax)
plt.show()
#Count plot of all catagorical data
def plot_catagorical(df):
    for feature in df.dtypes[df.dtypes == 'object'].index:
        sns.countplot(x=feature, data=df, order = df[feature].value_counts().index)
        plt.xticks(rotation=90)
        plt.show()
plot_catagorical(bank_data)
ROWS, COLS = 3, 4
fig, ax = plt.subplots(ROWS, COLS, figsize=(20, 15))
row, col = 0, 0
for i, categorical_feature in enumerate(categorical_columns):
    if col == COLS - 1:
        row += 1
    col = i % COLS
    bank_data[categorical_feature].value_counts().plot('bar', ax=ax[row, col])
#plot histogram of all numerical data
def plot_numerical(df):
    df.hist(figsize=(15,10))
    plt.show()
plot_numerical(bank_data)
#plot histogram of all numerical data
def plot_density_numerical(df):
    df.plot(kind='density', subplots=True, layout=(4,3), sharex=False,figsize=(15,10))
    plt.show()
plot_density_numerical(bank_data)<jupyter_output><empty_output><jupyter_text>Handling Outliers<jupyter_code>#plot box of all numerical data
def plot_box_numerical(df):
    df.plot(kind='box', subplots=True, layout=(4,3), sharex=False,figsize=(15,10))
    plt.show()
plot_box_numerical(bank_data)<jupyter_output><empty_output><jupyter_text>Function to find outliers<jupyter_code>def find_outliers_tukey(x):
    q1 = x.quantile(.25)
    q3 = x.quantile(.75)
    iqr = q3 - q1
    floor = q1 - 1.5*iqr
    ceiling = q3 + 1.5*iqr
    outlier_indices = list(x.index[(x < floor) | (x > ceiling)])
    outlier_values = list(x[outlier_indices])
    return outlier_indices, outlier_values<jupyter_output><empty_output><jupyter_text>Outliers Found in each numerical columns<jupyter_code>def outliers(columns):
    for column in columns:
        x, y = find_outliers_tukey(bank_data[column])
        print("Outliers for",column)
        print(np.sort(y))
outliers(numerical_columns)<jupyter_output>Outliers for age
[70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70
 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 70 71
 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71
 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71
 71 71 71 71 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72
 72 72 72 72 72 72 72 72 72 72 72 72 72 72 73 73 73 73 73 73 73 73 73 73
 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73 73
 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74
 74 74 74 74 74 74 74 74 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75
 75 75 75 75 75 75 75 75 76 76 76 76 76 76 76 76 76 76 76 76 76 76 76 76
 76 76 76 76 76 76 76 76 76 76 76 76 76 76 76 76 76 76 77 77 77 77 77 77
 77 77 77 77 77 77 77 77 77 77 77 77 77 77 78 78 78 78 78 78 78 78 78 78
 78 78 78 78 78 78 78 78 78 78 78 78 78 78 78 78 78 79 79 79 79 79 79 79
 79 79 79 79 79 79 79 80 80 80 80 [...]<jupyter_text>**As we can see from above `age`,`duration`,`campaign`, and `cons.price.idx` have many outliers**  Deleting rows<jupyter_code>#Dropping cons.price.index outliers
cons_index, cons_value = find_outliers_tukey(bank_data["cons.price.idx"])
bank_data = bank_data.drop(cons_index)<jupyter_output><empty_output><jupyter_text>Replacing with the max, min or median<jupyter_code>#replacing duration outliers with maximum value
max_ = bank_data['duration'].max()
bank_data['duration'] = np.where(bank_data.duration > 645,max_,bank_data['duration'])

#replacing campaign outliers with maximum value
max_ = bank_data['campaign'].max()
bank_data['campaign'] = np.where(bank_data.campaign > 7, max_,bank_data['campaign'])<jupyter_output><empty_output><jupyter_text>Target variable Distribution<jupyter_code>target = 'y'
sns.countplot(x=target,data=bank_data)
plt.title('subscribed')
plt.show()<jupyter_output><empty_output><jupyter_text>Preproccesing Data  Handling invalid data<jupyter_code>#dropping unknown values in job aatribute
bank_data = bank_data[bank_data.job != 'unknown']

#dropping unknown values in education attribute
bank_data = bank_data[bank_data.education != 'unknown']

#dropping unknown values in load attribute
bank_data = bank_data[bank_data.loan != 'unknown']

#dropping unknown values in housing attribute
bank_data = bank_data[bank_data.housing != 'unknown']

#dropping unknown values in default attribute
bank_data = bank_data[bank_data.default != 'unknown']

#dropping unknown values in marital attribute
bank_data = bank_data[bank_data.marital != 'unknown']
bank_data.info()
#percentage distribution of deposit
bank_data['y'].value_counts()/bank_data['y'].count()
bank_data['y'].value_counts()<jupyter_output><empty_output><jupyter_text>Handling Categorical Variables  Handling ordinal variables<jupyter_code>bank_data['education'].value_counts()
#education encoding with integer encoding 
level_mapping = {'illiterate': 0, 'basic.4y': 1, 'basic.6y': 2, 'basic.9y':3, 'high.school':4,'professional.course':5,
                 'university.degree': 6}
bank_data['education'] = bank_data['education'].replace(level_mapping)<jupyter_output><empty_output><jupyter_text>Handling Binary Features<jupyter_code># apply le on categorical feature columns

# encoding the binary feature
encoder = LabelEncoder()
categorical_cols = ['default','housing','loan','contact','y']
bank_data[categorical_cols] = bank_data[categorical_cols].apply(lambda col: encoder.fit_transform(col))
bank_data[categorical_cols].head()<jupyter_output><empty_output><jupyter_text>Handling nominal Variables<jupyter_code># transform data
bank_data = pd.get_dummies(bank_data, columns=['job','marital','month','day_of_week','poutcome'],
                                     drop_first=True)
bank_data.head()
bank_data.columns<jupyter_output><empty_output><jupyter_text>Correlation Matrix<jupyter_code>cor = bank_data.corr()
f = plt.figure(figsize=(19, 15))
plt.matshow(cor, fignum=f.number)
plt.xticks(range(bank_data.shape[1]), bank_data.columns, fontsize=14, rotation=45)
plt.yticks(range(bank_data.shape[1]), bank_data.columns, fontsize=14)
cb = plt.colorbar()
cb.ax.tick_params(labelsize=14)
plt.title('Correlation Matrix', fontsize=16);
bank_data.corrwith(bank_data.y).plot.bar(
        figsize = (20, 10), title = "Correlation with Deposit", fontsize = 20,
        rot = 45, grid = True)
plt.show()<jupyter_output><empty_output><jupyter_text>Balancing Data by upsampling small data<jupyter_code># Separate majority and minority classes
deposit_no = bank_data[bank_data.y==0]
deposit_yes= bank_data[bank_data.y==1]
print(deposit_no.shape,deposit_yes.shape)
#up sample deposit yes
from sklearn.utils import resample
deposit_yes_upsampled = resample(deposit_yes, replace=True, n_samples=len(deposit_no), random_state=0)
upsampled_bank_data = pd.concat([deposit_no, deposit_yes_upsampled])
sns_plot =sns.countplot(x='y',data=upsampled_bank_data)
plt.show()<jupyter_output><empty_output><jupyter_text>Split Data into Training and Test Sets<jupyter_code>#spliting data to train and test with ratio of 9 to 1
from sklearn.model_selection import train_test_split
features = ['age', 'education', 'default', 'housing', 'loan', 'contact', 'duration',
       'campaign', 'pdays', 'previous', 'emp.var.rate', 'cons.price.idx',
       'cons.conf.idx', 'euribor3m', 'nr.employed','job_blue-collar',
       'job_entrepreneur', 'job_housemaid', 'job_management', 'job_retired',
       'job_self-employed', 'job_services', 'job_student', 'job_technician',
       'job_unemployed', 'marital_married', 'marital_single', 'month_aug',
       'month_dec', 'month_jul', 'month_jun', 'month_mar', 'month_may',
       'month_nov', 'month_oct', 'month_sep', 'day_of_week_mon',
       'day_of_week_thu', 'day_of_week_tue', 'day_of_week_wed',
       'poutcome_nonexistent', 'poutcome_success']

X = bank_data[features]
y = bank_data[['y']]

X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.1, random_state=0)
from sklearn.preprocessing import MinMaxScaler
# define min max scaler
scaler = MinMaxScaler()
# transform data
columns_to_scale= ['age','campaign','cons.conf.idx','cons.price.idx','duration','emp.var.rate','euribor3m','nr.employed',
                   'pdays','previous']
X[columns_to_scale]  = scaler.fit_transform(X[columns_to_scale])
X[columns_to_scale].head()<jupyter_output><empty_output><jupyter_text>Normalization and Reduction  Rescale all numerical columns<jupyter_code>from sklearn.preprocessing import MinMaxScaler
# define min max scaler
scaler = MinMaxScaler()
# transform data
columns_to_scale= ['age','campaign','cons.conf.idx','cons.price.idx','duration','emp.var.rate','euribor3m','nr.employed',
                   'pdays','previous']
X_train[columns_to_scale]  = scaler.fit_transform(X_train[columns_to_scale])
X_test[columns_to_scale]  = scaler.fit_transform(X_test[columns_to_scale])<jupyter_output>C:\Users\owner pc\Anaconda3\lib\site-packages\sklearn\preprocessing\data.py:334: DataConversionWarning: Data with input dtype int64, float64 were all converted to float64 by MinMaxScaler.
  return self.partial_fit(X, y)
C:\Users\owner pc\Anaconda3\lib\site-packages\ipykernel_launcher.py:7: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  import sys
C:\Users\owner pc\Anaconda3\lib\site-packages\pandas\core\indexing.py:543: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  self.obj[item] = s
C:\Users\owner pc\Anaconda3\lib\site-packages\sklearn\prep[...]<jupyter_text>Import and Apply PCA<jupyter_code>from sklearn.decomposition import PCA
# Make an instance of the Model
pca = PCA()

pca.fit(X_train)
pca.explained_variance_ratio_
pca.n_components_
plt.figure(figsize=(12,9))
plt.plot(range(1,43),pca.explained_variance_ratio_.cumsum(), marker='o',linestyle='--')
plt.title("Explained variable by Components")
plt.xlabel("Number of Components")
plt.ylabel("Cumulativie Explained Varaince")
pca = PCA().fit(X_train)

% matplotlib inline
import matplotlib.pyplot as plt
plt.rcParams["figure.figsize"] = (12,6)

fig, ax = plt.subplots()
xi = np.arange(1, 43, step=1)
y = np.cumsum(pca.explained_variance_ratio_)

plt.ylim(0.0,1.1)
plt.plot(xi, y, marker='o', linestyle='--', color='b')

plt.xlabel('Number of Components')
plt.xticks(np.arange(0, 43, step=1)) #change from 0-based array index to 1-based human-readable label
plt.ylabel('Cumulative variance (%)')
plt.title('The number of components needed to explain variance')

plt.axhline(y=0.99, color='r', linestyle='-')
plt.text(0.5, 0.85, '99% cut-off threshold', color = 'red', fontsize=16)

ax.grid(axis='x')
plt.show()<jupyter_output><empty_output><jupyter_text>**Based on Above plots I used components with cummulative variance of `0.99%`. In another word  to get 99% of variance explained I need 32 principal components.**<jupyter_code>pca = PCA(0.99)  # project from 42 to 32 dimensions
projected = pca.fit_transform(X_train)
print(X_train.shape)
print(projected.shape)<jupyter_output>(27439, 42)
(27439, 32)<jupyter_text>Apply the mapping (transform) to both the training set and the test set.<jupyter_code>X_train = pca.fit_transform(X_train)
X_test = pca.fit_transform(X_test)<jupyter_output><empty_output><jupyter_text>Models   Logistic Regression Model<jupyter_code>print(X.shape,y.shape)<jupyter_output>(30488, 42) (30488, 1)<jupyter_text>Stratifid KFold<jupyter_code>from sklearn.model_selection import cross_val_score
from sklearn.model_selection import StratifiedKFold
from sklearn.pipeline import Pipeline
from sklearn.decomposition import PCA
from sklearn.linear_model import LogisticRegression
# define the pipeline
steps = [('pca', PCA(n_components=25)), ('m', LogisticRegression())]
model = Pipeline(steps=steps)
# evaluate model
cv = StratifiedKFold(n_splits=10, random_state=45)
n_scores = cross_val_score(model, X, y, scoring='accuracy', cv=cv, n_jobs=-1, error_score='raise')
scores = ['accuracy','f1','precision','recall','roc_auc']
all_scores =cross_validate(model, X, y, scoring=scores, cv=cv, n_jobs=-1, error_score='raise') 
# report performance
print('Accuracy: %.3f ' % (np.mean(all_scores['test_accuracy'])))
print('F1 Score: %.3f ' % (np.mean(all_scores['test_f1'])))
print('Precision: %.3f ' % (np.mean(all_scores['test_precision'])))
print('Recall: %.3f ' % (np.mean(all_scores['test_recall'])))
print('ROC AUC score: %.3f ' % (np.mean(all_scores['test_roc_auc'])))<jupyter_output>Accuracy: 0.823 
F1 Score: 0.271 
Precision: 0.387 
Recall: 0.282 
ROC AUC score: 0.867<jupyter_text>K Fold<jupyter_code>from sklearn.model_selection import KFold 
# define the pipeline
steps = [('pca', PCA(n_components=25)), ('m', LogisticRegression())]
model = Pipeline(steps=steps)
# evaluate model
cv = KFold(n_splits=10, random_state=45)
n_scores = cross_val_score(model, X, y, scoring='accuracy', cv=cv, n_jobs=-1, error_score='raise')
scores = ['accuracy','f1','precision','recall','roc_auc']
all_scores =cross_validate(model, X, y, scoring=scores, cv=cv, n_jobs=-1, error_score='raise') 
# report performance
print('Accuracy: %.3f ' % (np.mean(all_scores['test_accuracy'])))
print('F1 Score: %.3f ' % (np.mean(all_scores['test_f1'])))
print('Precision: %.3f ' % (np.mean(all_scores['test_precision'])))
print('Recall: %.3f ' % (np.mean(all_scores['test_recall'])))
print('ROC AUC score: %.3f ' % (np.mean(all_scores['test_roc_auc'])))<jupyter_output>Accuracy: 0.881 
F1 Score: 0.253 
Precision: 0.476 
Recall: 0.195 
ROC AUC score: 0.839<jupyter_text>XGBoost   Stratified KFold<jupyter_code>from sklearn.model_selection import cross_val_score
from sklearn.model_selection import StratifiedKFold
from sklearn.pipeline import Pipeline
from sklearn.decomposition import PCA
from xgboost import XGBClassifier
# define the pipeline
steps = [('pca', PCA(n_components=25)), ('m', XGBClassifier())]
model = Pipeline(steps=steps)
# evaluate model
cv = StratifiedKFold(n_splits=10, random_state=45)
n_scores = cross_val_score(model, X, y, scoring='accuracy', cv=cv, n_jobs=-1, error_score='raise')
# report performance
scores = ['accuracy','f1','precision','recall','roc_auc']
all_scores =cross_validate(model, X, y, scoring=scores, cv=cv, n_jobs=-1, error_score='raise') 
# report performance
print('Accuracy: %.3f ' % (np.mean(all_scores['test_accuracy'])))
print('F1 Score: %.3f ' % (np.mean(all_scores['test_f1'])))
print('Precision: %.3f ' % (np.mean(all_scores['test_precision'])))
print('Recall: %.3f ' % (np.mean(all_scores['test_recall'])))
print('ROC AUC score: %.3f ' % (np.mean(all_scores['test_roc_auc'])))<jupyter_output>Accuracy: 0.735 
F1 Score: 0.147 
Precision: 0.269 
Recall: 0.209 
ROC AUC score: 0.652<jupyter_text>KFold<jupyter_code># define the pipeline
steps = [('pca', PCA(n_components=25)), ('m', XGBClassifier())]
model = Pipeline(steps=steps)
# evaluate model
cv = KFold(n_splits=10, random_state=45)
n_scores = cross_val_score(model, X, y, scoring='accuracy', cv=cv, n_jobs=-1, error_score='raise')
# report performance
scores = ['accuracy','f1','precision','recall','roc_auc']
all_scores =cross_validate(model, X, y, scoring=scores, cv=cv, n_jobs=-1, error_score='raise') 
# report performance
print('Accuracy: %.3f ' % (np.mean(all_scores['test_accuracy'])))
print('F1 Score: %.3f ' % (np.mean(all_scores['test_f1'])))
print('Precision: %.3f ' % (np.mean(all_scores['test_precision'])))
print('Recall: %.3f ' % (np.mean(all_scores['test_recall'])))
print('ROC AUC score: %.3f ' % (np.mean(all_scores['test_roc_auc'])))<jupyter_output>Accuracy: 0.879 
F1 Score: 0.382 
Precision: 0.473 
Recall: 0.332 
ROC AUC score: 0.875<jupyter_text>Multilayer perceptron  Stratified KFold<jupyter_code>from sklearn.model_selection import cross_val_score, cross_validate
from sklearn.model_selection import StratifiedKFold
from sklearn.pipeline import Pipeline
from sklearn.decomposition import PCA
from sklearn.neural_network import MLPClassifier
# define the pipeline
steps = [('pca', PCA(n_components=30)), ('m', MLPClassifier(hidden_layer_sizes=(8,8,8), activation='relu',
                                                            solver='adam', max_iter=500))]
model = Pipeline(steps=steps)
# evaluate model
cv = StratifiedKFold(n_splits=10, random_state=45)
scores = ['accuracy','f1','precision','recall','roc_auc']
all_scores =cross_validate(model, X, y, scoring=scores, cv=cv, n_jobs=-1, error_score='raise') 
# report performance
print('Accuracy: %.3f ' % (np.mean(all_scores['test_accuracy'])))
print('F1 Score: %.3f ' % (np.mean(all_scores['test_f1'])))
print('Precision: %.3f ' % (np.mean(all_scores['test_precision'])))
print('Recall: %.3f ' % (np.mean(all_scores['test_recall'])))
print('ROC AUC score: %.3f ' % (np.mean(all_scores['test_roc_auc'])))<jupyter_output>Accuracy: 0.738 
F1 Score: 0.156 
Precision: 0.261 
Recall: 0.226 
ROC AUC score: 0.670<jupyter_text>KFold<jupyter_code>from sklearn.model_selection import KFold 
# define the pipeline
steps = [('pca', PCA(n_components=30)), ('m', MLPClassifier(hidden_layer_sizes=(8,8,8), activation='relu',
                                                            solver='adam', max_iter=500))]
model = Pipeline(steps=steps)
# evaluate model
cv = KFold(n_splits=10, random_state=45)
scores = ['accuracy','f1','precision','recall','roc_auc']
all_scores =cross_validate(model, X, y, scoring=scores, cv=cv, n_jobs=-1, error_score='raise') 
# report performance
print('Accuracy: %.3f ' % (np.mean(all_scores['test_accuracy'])))
print('F1 Score: %.3f ' % (np.mean(all_scores['test_f1'])))
print('Precision: %.3f ' % (np.mean(all_scores['test_precision'])))
print('Recall: %.3f ' % (np.mean(all_scores['test_recall'])))
print('ROC AUC score: %.3f ' % (np.mean(all_scores['test_roc_auc'])))<jupyter_output>Accuracy: 0.887 
F1 Score: 0.361 
Precision: 0.526 
Recall: 0.327 
ROC AUC score: 0.877
{'fraction_non_alphanumeric': 0.1344932462653189, 'fraction_numerical': 0.058368369263798195, 'mean_word_length': 5.292991837883479, 'pattern_counts': {'":': 0, '<': 110, '<?xml version=': 0, '>': 112, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '30', 'dir': 'jupyter-structured-clean-dedup', 'id': '218325', 'license': 'MIT', 'n_tokens_mistral': 9005, 'n_tokens_neox': 7696, 'n_words': 1916, 'path': 'notebook/Bank Institution Term Deposit.ipynb', 'repo_name': 'GETUCHALA20/Bank-Institution-Term-Deposit-Prediction', 'size': '22358'}