<jupyter_start><jupyter_text>Load the data and perform EDA.https://www.kaggle.com/pavansubhasht/ibm-hr-analytics-attrition-dataset1. Evaluate missing values2. Assess target class distribution3. Assess information value of individual features (correlation analysis and pairlot).<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

ibm = pd.read_csv('WA_Fn-UseC_-HR-Employee-Attrition.csv',index_col=0)

# Evaluate missing values
ibm.isnull().sum()
ibm.describe().transpose()
# Change data types for categorical variables
# Dummy code categorical features

#  Recoding 

ibm['BusinessTravel'][ibm['BusinessTravel'] == 'Non-Travel'] = 'Never'
ibm['BusinessTravel'][ibm['BusinessTravel'] == 'Travel_Rarely'] = 'Rarely'
ibm['BusinessTravel'][ibm['BusinessTravel'] == 'Travel_Frequently'] = 'Frequently'



ibm['Attrition'].replace('No',0,inplace=True)
ibm['Attrition'].replace('Yes',1,inplace=True)

ibm = pd.get_dummies(ibm)

ibm.info()
# Accessing target varaible distribution

print(ibm['Attrition'].mean())
ibm['Attrition'].hist(xrot=45.0)
# Pair Plot

from IPython.display import Image
import seaborn as sns
import matplotlib.pyplot as plt 

sns_plot = sns.pairplot(ibm, hue = 'Attrition')
sns_plot.savefig("pairplot.png")

plt.clf() # Clean parirplot figure from sns 

Image(filename='pairplot.png') # Show pairplot as image
# Correlation Analysis 

sns.heatmap(ibm.corr(), cmap="Spectral")
# Correlation Analysis 

ibm.corr()['Attrition'].sort_values(ascending=False)<jupyter_output><empty_output><jupyter_text>4. Pre-process the dataset5. Split the data into training/test datasets (70/30)4 pts.<jupyter_code>#Dropping variables

# ibm.drop(['Over18_Y'], axis=1, inplace=True)
# ibm.drop(['EmployeeCount'], axis=1, inplace=True)
# ibm.drop(['StandardHours'], axis=1, inplace=True)

# Preparing features and labels

X = ibm.drop('Attrition',axis=1).values
y = ibm['Attrition'].values

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.3,random_state=1)
from sklearn.preprocessing import MinMaxScaler

scaler = MinMaxScaler()
scaler.fit(X_train)

X_train = scaler.transform(X_train)
X_test = scaler.transform(X_test)<jupyter_output><empty_output><jupyter_text>6. Build a sequential neural network with the following parameters: 3 hidden dense layers - 100, 50, 25 nodes respectively, activation function = 'relu', dropout = 0.5 for each layer).7. Use early stopping callback to prevent overfitting.<jupyter_code>import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense,Activation,Dropout
model = Sequential()

model.add(Dense(units=100,activation='relu'))
model.add(Dense(units=50,activation='relu'))
model.add(Dense(units=25,activation='relu'))
model.add(Dense(units=1,activation='sigmoid'))
# For a binary classification problem
model.compile(loss='binary_crossentropy', optimizer='adam')
model.fit(x=X_train, 
          y=y_train,
          batch_size=128, 
          epochs=100,
          validation_data=(X_test, y_test), verbose=1
          )<jupyter_output>Epoch 1/100
9/9 [==============================] - 0s 12ms/step - loss: 0.6206 - val_loss: 0.5047
Epoch 2/100
9/9 [==============================] - 0s 3ms/step - loss: 0.4449 - val_loss: 0.4384
Epoch 3/100
9/9 [==============================] - 0s 3ms/step - loss: 0.4050 - val_loss: 0.4446
Epoch 4/100
9/9 [==============================] - 0s 3ms/step - loss: 0.3983 - val_loss: 0.4251
Epoch 5/100
9/9 [==============================] - 0s 3ms/step - loss: 0.3836 - val_loss: 0.4141
Epoch 6/100
9/9 [==============================] - 0s 3ms/step - loss: 0.3731 - val_loss: 0.4061
Epoch 7/100
9/9 [==============================] - 0s 3ms/step - loss: 0.3626 - val_loss: 0.3982
Epoch 8/100
9/9 [==============================] - 0s 3ms/step - loss: 0.3526 - val_loss: 0.3913
Epoch 9/100
9/9 [==============================] - 0s 3ms/step - loss: 0.3477 - val_loss: 0.3883
Epoch 10/100
9/9 [==============================] - 0s 3ms/step - loss: 0.3405 - val_loss: 0.3834
Epoch 11/100
9/9 [==========[...]<jupyter_text>8. Plot training and validation losses versus epochs.9. Print out model confusion matrix.10. Print out model classification report.11. Print out model ROC AUC.<jupyter_code>model_loss = pd.DataFrame(model.history.history)
model_loss.plot()
# with Dropout

from tensorflow.keras.layers import Dropout

model = Sequential()
model.add(Dense(units=100,activation='relu'))
model.add(Dropout(0.5))

model.add(Dense(units=50,activation='relu'))
model.add(Dropout(0.5))

model.add(Dense(units=25,activation='relu'))
model.add(Dropout(0.5))

model.add(Dense(units=1,activation='sigmoid'))

model.compile(loss='binary_crossentropy', optimizer='adam')
model.fit(x=X_train, 
          y=y_train, 
          batch_size=128,
          epochs=200,
          validation_data=(X_test, y_test), verbose=1,
          callbacks=[early_stop]
          )
model_loss = pd.DataFrame(model.history.history)
model_loss.plot()
y_pred = model.predict_classes(X_test)
from sklearn.metrics import classification_report,confusion_matrix, roc_auc_score

print(classification_report(y_test,y_pred))
print(confusion_matrix(y_test,y_pred))
print('ROC AUC: ', roc_auc_score(y_test,y_pred))<jupyter_output>[[363   1]
 [ 72   5]]
ROC AUC:  0.531093906093906
{'fraction_non_alphanumeric': 0.18678054214630524, 'fraction_numerical': 0.05402896398069068, 'mean_word_length': 5.513905683192261, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '668232', 'license': 'MIT', 'n_tokens_mistral': 2119, 'n_tokens_neox': 1875, 'n_words': 390, 'path': 'Notebook/Employee_Attrition_Prediction.ipynb', 'repo_name': 'rjparkk/rjparkk.github.io', 'size': '5386'}