<jupyter_start><jupyter_text>lets see the number of employees who left the company and those who didn't<jupyter_code>hr['left'].value_counts()
hr.groupby('left').mean()
hr.groupby('department').mean()
hr.groupby('salary').mean() 
%matplotlib inline
import matplotlib.pyplot as plt
pd.crosstab(hr.department, hr.left).plot(kind='bar')
plt.title('Turnover Frequency for Department')
plt.xlabel('Department')
plt.ylabel('Frequency of Turnover')
plt.savefig('department_bar_chart')
table = pd.crosstab(hr.salary, hr.left)
print(table)
table.div(table.sum(1).astype(float), axis=0).plot(kind='bar' ,stacked=True)
# axis='index'
plt.title('Stacked Bar Chart of Salary Level vs Turnover')
plt.xlabel('Salary Level')
plt.ylabel('Proportion of Employees')
plt.savefig('salary_bar_chart')<jupyter_output>left       0     1
salary            
high    1155    82
low     5144  2172
medium  5129  1317<jupyter_text>Histogram of numeric variables<jupyter_code>num_bins = 10

hr.hist(bins=num_bins, figsize=(20,15))
plt.savefig("hr_histogram_plots")
plt.show()
cat_vars = ['department', 'salary']
for var in cat_vars:
    cat_list = 'var' + '_' + var
    print(cat_list)
    cat_list = pd.get_dummies(hr[var], prefix=var)
    print(cat_list)
    hr1 = hr.join(cat_list)
    hr = hr1
hr.columns.tolist()
hr.drop(hr.columns[[8,9]], axis=1, inplace=True)
hr.columns.values<jupyter_output><empty_output><jupyter_text>so here our outcome variable is "left" i.e to be predicted and the others variables are predictors<jupyter_code>hr_vars = hr.columns.values.tolist()
y = ['left']
X=[i for i in hr_vars if i not in y]
X<jupyter_output><empty_output><jupyter_text>RFE is Recursive feature elemination. Help us identify which columns or variables will contribute the most to predicting the target attribute<jupyter_code>from sklearn.feature_selection import RFE
from sklearn.linear_model import LogisticRegression

model = LogisticRegression()

rfe = RFE(model, 10)
rfe = rfe.fit(hr[X], hr[y])
print(rfe.support_)
print(rfe.ranking_)
z = rfe.ranking_
required_cols = []
for i,j in zip(rfe.ranking_,X):
    if i == 1:
        required_cols.append(j)
X = hr[required_cols]
y = hr['left']<jupyter_output><empty_output><jupyter_text>Logistic Regression Model<jupyter_code>from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X,y, test_size=0.3, random_state=0)

from sklearn.linear_model import LogisticRegression
from sklearn import metrics
logreg = LogisticRegression()
logreg.fit(X_train, y_train)
metrics.accuracy_score(y_test, logreg.predict(X_test))
print('Logistic regression accuracy: {:.3f}'.format(metrics.accuracy_score(y_test, logreg.predict(X_test))))<jupyter_output>Logistic regression accuracy: 0.771<jupyter_text>Random Forest<jupyter_code>from sklearn.ensemble import RandomForestClassifier
rf = RandomForestClassifier()
rf.fit(X_train, y_train)
print('Random Forest Classifier Accuracy: {:.3f}'.format(metrics.accuracy_score(y_test, rf.predict(X_test))))<jupyter_output>Random Forest Classifier Accuracy: 0.976<jupyter_text>SVM<jupyter_code>from sklearn.svm import SVC
svc = SVC()
svc.fit(X_train, y_train)
print('SVM accuracy is : {:.3f}'.format(metrics.accuracy_score(y_test, svc.predict(X_test))))<jupyter_output>SVM accuracy is : 0.909<jupyter_text>Cross Validation is important as it seems Random Forest is way better than other techniques Cross Validation attempts to avoid overfitting while still producing a prediction for each observation set<jupyter_code>from sklearn import model_selection
from sklearn.model_selection import cross_val_score
kfold = model_selection.KFold(n_splits=10, random_state=7)
modelCV = RandomForestClassifier()
scoring = 'accuracy'
results = model_selection.cross_val_score(modelCV ,X_train, y_train, cv=kfold, scoring=scoring)
print('10-fold cross validation average accuracy: %.3f' %(results.mean()))<jupyter_output>10-fold cross validation average accuracy: 0.979<jupyter_text>average accuracy is very close to Random Forest Model accuracy<jupyter_code>from sklearn.metrics import classification_report
print(classification_report(y_test, rf.predict(X_test)))
y_pred = rf.predict(X_test)
from sklearn.metrics import confusion_matrix
import seaborn as sns
forest_cm = metrics.confusion_matrix(y_pred, y_test, [1,0])
sns.heatmap(forest_cm, annot=True, fmt='.2f', xticklabels = ["Left", "Stayed"], yticklabels = ["Left", "Stayed"])
plt.ylabel('True Class')
plt.xlabel('Predicted Class')
plt.title('Random Forest')
plt.savefig('random_forest')<jupyter_output><empty_output><jupyter_text>When an employee left, how often does my classifier predict that correctly? This measurement is called “recall”  random forest correctly retrieved 991 out of 1038. This translates to a turnover “recall” of about 95% (991/1038), far better than logistic regression (26%) or support vector machines (85%).  Logistic Regression<jupyter_code>print(classification_report(y_test, logreg.predict(X_test)))
logreg_y_pred = logreg.predict(X_test)
logreg_cm = metrics.confusion_matrix(logreg_y_pred, y_test, [1,0])
sns.heatmap(logreg_cm, annot=True, fmt='.2f',xticklabels = ["Left", "Stayed"] , yticklabels = ["Left", "Stayed"] )
plt.ylabel('True class')
plt.xlabel('Predicted class')
plt.title('Logistic Regression')
plt.savefig('logistic_regression')<jupyter_output><empty_output><jupyter_text>Support Vector Machine<jupyter_code>print(classification_report(y_test, svc.predict(X_test)))
svc_y_pred = svc.predict(X_test)
svc_cm = metrics.confusion_matrix(svc_y_pred, y_test, [1,0])
sns.heatmap(svc_cm, annot=True, fmt='.2f',xticklabels = ["Left", "Stayed"] , yticklabels = ["Left", "Stayed"] )
plt.ylabel('True class')
plt.xlabel('Predicted class')
plt.title('Support Vector Machine')
plt.savefig('support_vector_machine')<jupyter_output><empty_output><jupyter_text>When a classifier predicts an employee will leave, how often does that employee actually leave? This measurement is called “precision”. Random forest again outperforms the other two at about 95% precision (991 out of 1038) with logistic regression at about 51% (273 out of 540), and support vector machine at about 77% (890 out of 1150(890+260)).<jupyter_code>from sklearn.metrics import roc_auc_score
from sklearn.metrics import roc_curve

logit_roc_auc = roc_auc_score(y_test, logreg.predict(X_test))
fpr, tpr, thresholds = roc_curve(y_test, logreg.predict_proba(X_test)[:,1])

rf_roc_auc = roc_auc_score(y_test, rf.predict(X_test))
rf_fpr, rf_tpr, rf_thresholds = roc_curve(y_test, rf.predict_proba(X_test)[:,1])

plt.figure()
plt.plot(fpr, tpr, label='Logistic Regression (area = %0.2f)' % logit_roc_auc)
plt.plot(rf_fpr, rf_tpr, label='Random Forest (area = %0.2f)' % rf_roc_auc)
plt.plot([0,1], [0,1], 'r--')
plt.xlim([0.0,1.0])
plt.ylim([0.0,1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receier operating characteristic')
plt.legend(loc="lower right")
plt.savefig('ROC')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12476722532588454, 'fraction_numerical': 0.021486892995272882, 'mean_word_length': 7.311904761904762, 'pattern_counts': {'":': 0, '<': 44, '<?xml version=': 0, '>': 44, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '128030', 'license': 'MIT', 'n_tokens_mistral': 2633, 'n_tokens_neox': 2458, 'n_words': 559, 'path': 'HR Project/HR EDA.ipynb', 'repo_name': 'sarsiz/Data_Science_Personal_Projects', 'size': '6981'}