<jupyter_start><jupyter_text>Data PreProcessing-<jupyter_code>#importing Required Libraries

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from datetime import datetime ,timedelta
#load the data
from google.colab import files                 #upload the data on the colab notebook
uploded = files.upload()
#store the data into dataframe
df= pd.read_csv('train_MpHjUjU.csv' )   # reading the data


#printing the first 10 rows
df.head(5)
df.shape   #checking no. of column & rows
df.dtypes  #checking datatypes of the column
#get count of the empty value of each column
df.isna().sum()
#check for any missing value
df.isnull().values.any()
#Checking  some statistics
df.describe()
#Deletig for  dublicate values
df = df.drop_duplicates(subset=['Emp_ID'	,'Age',	'Gender','City','Education_Level',	'Salary','Dateofjoining'],keep = 'last')
df.head()<jupyter_output><empty_output><jupyter_text>Feature Engineering   * Adding two new feature 1.   Attrition - From last working date clumn2.  Year = Number of years employee  worked in the company<jupyter_code>#adding one new column attrition form last ''last working date 'column
df['Attrition'] = df['LastWorkingDate']
df['Attrition'] = df['Attrition'].fillna(0)           #Replacing NAN values to 0 
y = pd.DataFrame(df['Attrition'], columns = ['attrition'])
y['attrition']= df['Attrition']
y['attrition'] = y['attrition'].str.isnumeric()

def datetime_to_int(y):
   return int(y['attrition'].strftime("%Y-%m-%d"))       #replace date to false value


y =y.replace(to_replace=False,value = 1)    #reaplce false value to the in 1 for all of those who have left the organisation

y= y.fillna(0)                             ##making new feature attrition from lastworkingday column 
y.head()
#Adding new feature attrition from lastworkingday column 
df['Attrition']=y['attrition']
df['Attrition'].value_counts()            

#the number of employees that stayed and left the company
sns.countplot(df['Attrition'])
df.shape<jupyter_output><empty_output><jupyter_text>*  We have unique 3786 data points and 14 column,Out of them 2170 data point belonging to Class NO(0) and 1616 data point belonging to class 'Yes(1)'<jupyter_code>plt.subplots(figsize=(12,4))
sns.countplot(x='Age',hue='Attrition',data=df, palette='colorblind')<jupyter_output><empty_output><jupyter_text>* From 33 onwards age group, most of the employees did not leave job and majority in attrition is contributed by age group of 31 & 32 . **Replacing NaN values to prediction date date so we can calculate number of years employee work**<jupyter_code>#filling Na/NaN values
 
df['LastWorkingDate'] = df['LastWorkingDate'].fillna('2018-01-01')  #filling nan values to prediction date date so we can calculate number of years employee work

df.head(5)<jupyter_output><empty_output><jupyter_text>**Introducing new feature using first working day and last working day variables**<jupyter_code>#Introducing new feature using first working day and last working day variables-

df["Dateofjoining"] = pd.to_datetime(df["Dateofjoining"])             #handling timeseries data
df["LastWorkingDate"] = pd.to_datetime(df["LastWorkingDate"])

df4 = pd.DataFrame( columns = ['day','year'])                           #creating  2 new features number of days employee work and number of years employee work
df4['day'] = df['LastWorkingDate'] - df['Dateofjoining']
df4['year'] = df4["day"] / timedelta(days=365)
print(df4)<jupyter_output>day      year
2      78 days  0.213699
4      56 days  0.153425
9     141 days  0.386301
12     58 days  0.158904
17    154 days  0.421918
...        ...       ...
19082 885 days  2.424658
19090 419 days  1.147945
19096 335 days  0.917808
19097 207 days  0.567123
19103 207 days  0.567123

[3786 rows x 2 columns]<jupyter_text>Dropping unwanted column from Dataset -<jupyter_code>df=df.drop(labels=['MMM-YY','Dateofjoining','LastWorkingDate','Gender','City','Education_Level'],axis=1)  # dropping the table because they are useless in our prediction

#adding  new features to our data
df['year'] = df4['year']
df.head()
df.to_csv('test matching data.csv')    # creating csv before dropping emp_id column so we can merge test csv with train csv for prediction

df.isna().sum()  #checking the na values<jupyter_output><empty_output><jupyter_text>**Dropping id table-**<jupyter_code>#dropping id table
df = df.drop (labels='Emp_ID',axis = 1)                 
df['attrition']= df['Attrition']
df= df.drop(labels='Attrition',axis = 1)    #shifting attrition table at last
df.head()<jupyter_output><empty_output><jupyter_text>Visualizing our Variables-<jupyter_code>#get the correlation
df.corr()

#visualize the correlation
plt.subplots(figsize=(12,5))
sns.heatmap(df.corr(), annot=True,fmt ='.0%')
df.to_csv('processedtrain.csv')    # processed trainnig data csv
df.shape  #checking shape of data<jupyter_output><empty_output><jupyter_text>Dividing the Data into Two Parts "TRAINING" and "TESTING" -<jupyter_code>#split the data
x = df.iloc[:,0:7].values      # splitting all the rows from column 0 to 6 in x
y = df.iloc[: ,7].values        #splitting attrtion column in y       

print('x{}'.format(x))
print('================================================')
print('y:{}'.format(y))<jupyter_output>x[[2.80000000e+01 5.73870000e+04 1.00000000e+00 ... 0.00000000e+00
  2.00000000e+00 2.13698630e-01]
 [3.10000000e+01 6.70160000e+04 2.00000000e+00 ... 0.00000000e+00
  1.00000000e+00 1.53424658e-01]
 [4.30000000e+01 6.56030000e+04 2.00000000e+00 ... 0.00000000e+00
  1.00000000e+00 3.86301370e-01]
 ...
 [2.80000000e+01 6.94980000e+04 1.00000000e+00 ... 0.00000000e+00
  1.00000000e+00 9.17808219e-01]
 [2.90000000e+01 7.02540000e+04 2.00000000e+00 ... 0.00000000e+00
  1.00000000e+00 5.67123288e-01]
 [3.00000000e+01 7.02540000e+04 2.00000000e+00 ... 4.11480000e+05
  2.00000000e+00 5.67123288e-01]]
================================================
y:[1. 0. 1. ... 1. 0. 0.]<jupyter_text>Building The model from "TRAINING DATA SET"-<jupyter_code>#split the data into 25% testing and 75% training
from sklearn.model_selection import train_test_split               #importing train_test_split
x_train, x_test, y_train, y_test = train_test_split(x,y,test_size = 0.20,train_size= 0.80  , random_state =0 )  #splitting  data , 80-20=train-test split
#using KNN Classifier as  our model
from sklearn.neighbors import KNeighborsClassifier          #importing knnClassifier 
knn =KNeighborsClassifier(n_neighbors=7,weights='uniform',algorithm='auto')  # n=7
knn.fit(x_train, y_train)     #fitting the model 

y_pred = knn.predict(x_test)   # predicting cross validation test data<jupyter_output><empty_output><jupyter_text>Evaluating KNNclassifier using macro f1_score metrics -<jupyter_code>from sklearn.metrics import f1_score   #importing f1_score from sklearn
f1 = f1_score(y_test,y_pred)    #getting the f1_score
print(f1)<jupyter_output>0.7378640776699029<jupyter_text>4. Test data: *   Creating test file from given 'test_hXY9mYw.csv'<jupyter_code>#Creating test file from given 'test_hXY9mYw.csv'
df1=pd.read_csv('/content/test matching data.csv')
df2=pd.read_csv('/content/test_hXY9mYw.csv')

test_data = pd.merge(df2,df1)   #merginf df1 and df2 using EMP_ID column

#dropping duplicate values
test_data = test_data.drop(labels=['Attrition','Unnamed: 0'],axis=1)
test_data=test_data.drop_duplicates(subset= 'Emp_ID',keep='last')
test_data.to_csv('test_data.csv')

test_data.shape<jupyter_output><empty_output><jupyter_text>Dropping the Emp_ID column from test data<jupyter_code>#dropping the Emp_ID column -
test_data= test_data.drop(labels='Emp_ID',axis=1)
test_data.head()<jupyter_output><empty_output><jupyter_text>**Predicting the given test-data point** :-<jupyter_code>Final_result= knn.predict(test_data)    #Predicting the given test-data point where  how many employee will leave the company in given jan-2018 quarter
#Creating submission file of the final result-

df6=pd.read_csv('/content/test_data.csv')
df6=df6.drop(labels='Unnamed: 0', axis=1)
test= pd.DataFrame(columns=['Emp_ID','Target'])
test['Emp_ID']=df6['Emp_ID']
test['Target']=Final_result
test.head(20)
test['Target'].value_counts() #checking predicted values_count<jupyter_output><empty_output><jupyter_text>Conclusion :- *  From above result we can assume that out of 741 employees ,we can expect that 160 employees will leave the company in the upcoming two quarters (01 Jan 2018 - 01 July 2018)  and  581 employees will be remain at their designation.    Suggestion:-*  As data scientist i would suggest From above Analysis and Prediction that ,company should offer more salary to more experience 1.   company should offer more salary to more experience person.2.  Company should give better designation to old employees.  Submission File :-<jupyter_code>test.to_csv('Final_result.csv')   #submission file<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12597008210549995, 'fraction_numerical': 0.0777190417275897, 'mean_word_length': 5.162162162162162, 'pattern_counts': {'":': 0, '<': 58, '<?xml version=': 0, '>': 58, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 23, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '509705', 'license': 'MIT', 'n_tokens_mistral': 3502, 'n_tokens_neox': 2979, 'n_words': 819, 'path': 'Employee_Attrition_notebook_.ipynb', 'repo_name': 'Tushar2771/Employee-Attrition-jobathon--21nov21', 'size': '8891'}