<jupyter_start><jupyter_text>Data Cleaning<jupyter_code># For Numerical Data
df.skew()<jupyter_output><empty_output><jupyter_text>In generalSkew value is less than -0.5 -> Left SkewedSkew value between -0.5 to 0.5 -> Normalized dataSkew value is greater than 0.5 -> Right SkewedThere are few values in the columns which have no values, which in turn leads to inconsistency during prediction. Thus, we replace these values and it is done as follows:1. Check for skewness of the particular feature.2. If it is normalized we replace it with the Mean value.3. If it is Right-skewed or Left-skewed we replace it with the Median value.<jupyter_code># Listing the columns according to the skewness
df1 = pd.DataFrame([df.skew()])
Normalized_Data=[]
Right_Skew=[]
Left_Skew=[]
for label in df.columns:
    if df1[label][0] < -0.5:
        Left_Skew.append(label)
    elif df1[label][0] <=0.5:
        Normalized_Data.append(label)
    else:
        Right_Skew.append(label)
# Prints columns name which are Left Skewed
Left_Skew
# Prints columns name which are Normal
Normalized_Data
# Prints columns name which are Right Skewed
Right_Skew
# Replacing NaN value on the basis of skewness

# For Normalized data
for label in Normalized_Data:
    df[label].fillna(round(df[label].mean()),inplace=True) # Round() function is used because some coloumns are binary so by rounding value will become integer

# For Left and Right Skewness
for label in Right_Skew+Left_Skew:
    df[label].fillna(df[label].median(),inplace=True)
# Crosschecks that number of null value in each column
df.isnull().sum()<jupyter_output><empty_output><jupyter_text>Now we can see that there are ni null value in any column  Data Visualisation<jupyter_code># Prints first five rows of dataset
df.head()
# Prints columns name of the dataset
df.columns
plt.figure(figsize=(15,6))
sns.countplot(df["MALE_DIV"])
plt.xlabel("Applicant is male and divorced(0:No, 1:Yes)", fontsize=15)
plt.ylabel("Count", fontsize=15)

df["MALE_DIV"].value_counts()<jupyter_output><empty_output><jupyter_text>Here we can see that very few applicant are male and divorced<jupyter_code>plt.figure(figsize=(15,6))
sns.countplot(df["MALE_SINGLE"])
plt.xlabel("Applicant is male and single(0:No, 1:Yes)", fontsize=15)
plt.ylabel("Count", fontsize=15)

df["MALE_SINGLE"].value_counts()<jupyter_output><empty_output><jupyter_text>We can see that there are more number of male applicant who are single<jupyter_code>plt.figure(figsize=(15,6))
sns.countplot(df["MALE_MAR_or_WID"])
plt.xlabel("Applicant is male and married or a widower(0:No, 1:Yes)", fontsize=15)
plt.ylabel("Count", fontsize=15)

df["MALE_MAR_or_WID"].value_counts()<jupyter_output><empty_output><jupyter_text>We can see that thare are very few applicant who are male or a winower<jupyter_code>plt.figure(figsize=(15,6))
sns.countplot(df["CO-APPLICANT"])
plt.xlabel("Application has a co-applicant(0:No, 1:Yes)", fontsize=15)
plt.ylabel("Count", fontsize=15)

df["CO-APPLICANT"].value_counts()<jupyter_output><empty_output><jupyter_text>We can see that there are very few application which has a co-appicant<jupyter_code>plt.figure(figsize=(15,6))
sns.countplot(df["JOB"])
plt.xlabel("Nature of JOB", fontsize=15)
plt.ylabel("Count", fontsize=15)

print("Nature of JOB\n**************\n0.0 : unemployed/ unskilled  - non-resident\n1.0 : unskilled - resident\n2.0 : skilled employee / official\n3.0 : management/ self-employed/highly qualified employee/ officer")
df["JOB"].value_counts()<jupyter_output>Nature of JOB
**************
0.0 : unemployed/ unskilled  - non-resident
1.0 : unskilled - resident
2.0 : skilled employee / official
3.0 : management/ self-employed/highly qualified employee/ officer<jupyter_text>We can see that there are most of the employee are skilled followed by unskilled and then followed by management and only few employee are unskilled<jupyter_code>plt.figure(figsize=(15,8))
df.groupby(["JOB","RESPONSE"]).size().plot(kind="bar")
plt.ylabel("Count", fontsize=15)
plt.xlabel("(Value1,Value2) where Value1 is Nature of JOB and Value2 is Credit Rating is good", fontsize=15)
print("Nature of JOB\n**************\n0.0 : unemployed/ unskilled  - non-resident\n1.0 : unskilled - resident\n2.0 : skilled employee / official\n3.0 : management/ self-employed/highly qualified employee/ officer\n\n\nCredit Rating is good\n*******************************\n0.0:No\n1.0:Yes")<jupyter_output>Nature of JOB
**************
0.0 : unemployed/ unskilled  - non-resident
1.0 : unskilled - resident
2.0 : skilled employee / official
3.0 : management/ self-employed/highly qualified employee/ officer


Credit Rating is good
*******************************
0.0:No
1.0:Yes<jupyter_text>We can see that most of the good credit rater are from skilled employee and very few are from unskilled employee<jupyter_code>plt.figure(figsize=(15,6))
sns.countplot(df["TELEPHONE"])
plt.xlabel("Applicant has phone in his or her name(0:No, 1:Yes)", fontsize=15)
plt.ylabel("Count", fontsize=15)

df["TELEPHONE"].value_counts()<jupyter_output><empty_output><jupyter_text>We can see that there are more number of applicant who does not have phone in his or her name<jupyter_code>plt.figure(figsize=(15,6))
sns.countplot(df["FOREIGN"])
plt.xlabel("Foreign worker(0:No, 1:Yes)", fontsize=15)
plt.ylabel("Count", fontsize=15)

df["FOREIGN"].value_counts()<jupyter_output><empty_output><jupyter_text>We can see that there are very few Foreign Worker<jupyter_code>plt.figure(figsize=(15,6))
sns.countplot(df["RESPONSE"])
plt.xlabel("Credit rating is good(0:No, 1:Yes)", fontsize=15)
plt.ylabel("Count", fontsize=15)

df["RESPONSE"].value_counts()<jupyter_output><empty_output><jupyter_text>We can see that there are more number of employee who has good credit rating  Creating Models<jupyter_code># Splitting the Dataset into training and testing part

# Features
x=df.drop("RESPONSE", axis=1)
# Target
y=df['RESPONSE']

x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=5)
print(x_train.shape)
print(y_train.shape)
print(x_test.shape)
print(y_test.shape)
# Standardising the data
# Before making any actual predictions, it is always a good practice to scale the features so that all of them can be uniformly evaluated

scaler = StandardScaler() 
scaler.fit(x_train)

x_train = scaler.transform(x_train)  
x_test = scaler.transform(x_test)
# prepare models
models = []
models.append(('LR', LogisticRegression()))
models.append(('LDA', LinearDiscriminantAnalysis()))
models.append(('KNN', KNeighborsClassifier()))
models.append(('CART', DecisionTreeClassifier()))
models.append(('NB', GaussianNB()))
models.append(('RF', RandomForestClassifier()))
models.append(('SVM', SVC(gamma='auto')))



for name, model in models:
    # Training the model
    model.fit(x_train,y_train)

    # Prediction
    y_pred=model.predict(x_test)
    
    print("Accuracy of {} is {:.2f} %" .format(name, model.score(x_test,y_test)*100))<jupyter_output>Accuracy of LR is 74.13 %
Accuracy of LDA is 72.64 %
Accuracy of KNN is 72.14 %
Accuracy of CART is 65.17 %
Accuracy of NB is 69.65 %
Accuracy of RF is 67.16 %
Accuracy of SVM is 73.63 %<jupyter_text>Tree-based Feature SelectionBagged decision trees like Random Forest and Extra Trees can be used to estimate the importance of features.In the example below we construct a ExtraTreesRegressor regressorUse ExtraTreesClassifier classifier for classification problemTree-based estimators can be used to compute feature importances, which in turn can be used to discard irrelevant features (when coupled with the sklearn.feature_selection.SelectFromModel meta-transformer)<jupyter_code># Feature Importance with Extra Trees Regressor
from pandas import read_csv
from sklearn.ensemble import ExtraTreesRegressor

# feature extraction
etr = ExtraTreesRegressor()
etr.fit(x, y)
etr_coef=etr.feature_importances_
print(etr_coef)
etr1_coef = pd.Series(etr_coef, index = x.columns)
etr1_coef<jupyter_output><empty_output><jupyter_text>You can see that we are given an importance score for each attribute where the larger score the more important the attribute.<jupyter_code>imp_coef = etr1_coef.sort_values()
plt.figure(figsize=(15, 10))
imp_coef.plot(kind = "barh")
plt.title("Feature importance using Extra Trees Regressor", fontsize=20)
from sklearn.feature_selection import SelectFromModel

sfm = SelectFromModel(etr, prefit=True)
x_n = sfm.transform(x)
x_n.shape
#Spliting the dataset into a training set and a testing set
x_train,x_test,y_train,y_test=train_test_split(x_n,y,test_size=0.2,random_state=5)
print(x_train.shape)
print(y_train.shape)
print(x_test.shape)
print(y_test.shape)

# Since the range of values of raw data varies widely, in some machine learning algorithms, objective functions will not work properly without normalization. 
scaler = StandardScaler()  
scaler.fit(x_train)

x_train = scaler.transform(x_train)  
x_test = scaler.transform(x_test)<jupyter_output>(801, 10)
(801,)
(201, 10)
(201,)<jupyter_text>Models with the selected features<jupyter_code># prepare models
models = []
models.append(('LR', LogisticRegression()))
models.append(('LDA', LinearDiscriminantAnalysis()))
models.append(('KNN', KNeighborsClassifier()))
models.append(('CART', DecisionTreeClassifier()))
models.append(('NB', GaussianNB()))
models.append(('RF', RandomForestClassifier()))
models.append(('SVM', SVC(gamma='auto')))


for name, model in models:
    # Training the model
    model.fit(x_train,y_train)

    # Prediction
    y_pred=model.predict(x_test)
    
    print("Accuracy of {} is {:.2f} %" .format(name, model.score(x_test,y_test)*100))
'''
Model        Accuracy Before Feature Selection          Accuracy After Feature Selection
*******************************************************************************************
 LR                     74.13 %                                   75.62 %
 LDA                    72.64 %                                   74.63 %
 KNN                    72.14 %                                   72.64 %
 CART                   65.17 %                                   59.20 %
 NB                     69.65 %                                   76.62 %
 RF                     67.16 %                                   67.16 %
 SVM                    73.63 %                                   74.63 %
 
 Accuracy of LR, LDA, KNN, NB, SVM increases after feature selection whereas Accuracy of CART decreases anf for RF it remains same.
 
 After feature selection NB gives high accuracy so NB is good algorithm for this dataset.
 
LR: Logistic Regression
LDA: Linear Discriminant Analysis
KNN: K Nearest Neighbors Classifier
CART: Decision Tree Classifier
NB: Naive Bayes
RF: Random Forest Classifier
SVM: Support Vector Machine
'''
print()<jupyter_output>
{'fraction_non_alphanumeric': 0.1271611823759063, 'fraction_numerical': 0.02463283138129764, 'mean_word_length': 4.784408602150537, 'pattern_counts': {'":': 0, '<': 58, '<?xml version=': 0, '>': 59, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '471322', 'license': 'MIT', 'n_tokens_mistral': 3747, 'n_tokens_neox': 3480, 'n_words': 978, 'path': 'Soulpage IT Solutions Pvt. Ltd/Soulpage IT Solutions Pvt. Ltd.ipynb', 'repo_name': 'mohansah/Work_Did_For_Companies', 'size': '10758'}