<jupyter_start><jupyter_text>Context and ContentA company which is active in Big Data and Data Science wants to hire data scientists among people who successfully pass some courses which conduct by the company. Many people signup for their training. Company wants to know which of these candidates are really wants to work for the company after training or looking for a new employment because it helps to reduce the cost and time as well as the quality of training or planning the courses and categorization of candidates. Information related to demographics, education, experience are in hands from candidates signup and enrollment.So this dataset designed to understand the factors that lead a person will work for the company , by model(s) that uses the current credentials/demographics/experience to predict the probability of a candidate to look for a new job or will work for the company.The whole data divided to train and test . Target isn't included in test but the target values is in hands for related tasks. I provided a sample submission corresponds to enrolleeid of test set too ( enrolleid - target) Also:The dataset is imbalancedMost features are categorical, some with high cardinality My Task:- Predict the probability of a candidate will work for the company- Interpret models such a way that illustrate which features affects candidate decision- This is a Supervised Classification Task  Read Data from Files<jupyter_code># For Dataframing
import pandas as pd
# For Numbering
import numpy as np
test = pd.read_csv('data/aug_test.csv')
train = pd.read_csv('data/aug_train.csv')
print('Test Shape')
print(test.shape)
print('Train Shape')
print(train.shape)
train.head()
train.describe()
# Examine Target
train['target'].value_counts()
# Percent Positive
print("Percent Positive Targets:")
percent_pos = train['target'].value_counts()[1]/len(train['target'])
percent_pos
train.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 19158 entries, 0 to 19157
Data columns (total 14 columns):
 #   Column                  Non-Null Count  Dtype  
---  ------                  --------------  -----  
 0   enrollee_id             19158 non-null  int64  
 1   city                    19158 non-null  object 
 2   city_development_index  19158 non-null  float64
 3   gender                  14650 non-null  object 
 4   relevent_experience     19158 non-null  object 
 5   enrolled_university     18772 non-null  object 
 6   education_level         18698 non-null  object 
 7   major_discipline        16345 non-null  object 
 8   experience              19093 non-null  object 
 9   company_size            13220 non-null  object 
 10  company_type            13018 non-null  object 
 11  last_new_job            18735 non-null  object 
 12  training_hours          19158 non-null  int64  
 13  target                  19158 non-null  float64
dtypes: float64(2), int64(2), object(10)
me[...]<jupyter_text>Dealing with Nans<jupyter_code># Label Nans in Categorical Variables
def fill_categorical_nans(df):
    cat_columns = df.describe(exclude='number').columns
    for col in cat_columns:
        df[col] = df[col].fillna(str(col) + '_nan')
    
    return df

train = fill_categorical_nans(train)

from sklearn.impute import SimpleImputer
imputer = SimpleImputer(strategy='median')
numerical_cols = train.select_dtypes([np.number]).columns
train[numerical_cols] = imputer.fit_transform(train[numerical_cols])

assert train.isna().sum().sum() == 0, 'There are Nans in Data Frame.'<jupyter_output><empty_output><jupyter_text>Examining Feature Distributions<jupyter_code>train.describe()
# Plot Numerical Variables
%matplotlib inline
import matplotlib.pyplot as plt
train.drop('enrollee_id', axis=1).hist(bins=50, figsize=(20, 15))
plt.show()<jupyter_output><empty_output><jupyter_text>Examining Cardinality of Non-Numeric Columns<jupyter_code>train.describe(exclude='number')
import helpers.cardinality as crd
crd.cardinality_report(train)
# Categorical Variables Ranked from Low to High Cardinality
ranked = crd.rank_cardinality(train)
low_crd = ranked[:-1]
low_crd
# Drop Enrollee ID, City -> Cardinality too high
# T- Try to see if model performance improves when including city. -T
# -- Cities were showing up in the val set that weren't in the train set...
# --- and this messed up the encoding step
# --- T- Maybe try to 'stratify sample' the train set so that it includes all cities? -T
# --- T- Or put least common cities into one category called 'Uncommon_City'? -T
train = train.drop(['enrollee_id', 'city'], axis=1)
len(train.columns)
# Plot Categorical Variables
%matplotlib inline
import matplotlib.pyplot as plt
def draw_histograms(df, variables, n_rows, n_cols):
    fig=plt.figure(figsize=(15, 20))
    for i, var_name in enumerate(variables):
        ax=fig.add_subplot(n_rows,n_cols,i+1)
        df[var_name].hist(bins=10,ax=ax)
        ax.set_title(var_name+" Distribution")
    fig.tight_layout()  # Improves appearance a bit.
    plt.show()

draw_histograms(train, low_crd, 5, 2)<jupyter_output><empty_output><jupyter_text>Separate Train into Train and Val<jupyter_code># Separate Val and Train
from sklearn.model_selection import train_test_split

train, val = train_test_split(train, test_size=0.5, random_state=42)
print("Train: ", train.shape)
print("Val: ", val.shape)
# Separate X, y
X_train = train.drop('target', axis=1)
y_train = train['target']
X_val = val.drop('target', axis=1)
y_val = val['target']
print(X_train.shape, y_train.shape, X_val.shape, y_val.shape)<jupyter_output>(9579, 11) (9579,) (9579, 11) (9579,)<jupyter_text>Baseline Prediction<jupyter_code># Establish Baseline
val['target'].value_counts()
# Looks like most respondents don't want to change jobs.
# Baseline predicts all respondents to be non-changers.
base_pred = [0.0] * (val.shape[0] - 1)
base_pred.append(1.0) # Recall is 0 if there are 0 True Pos 
                      # -> So insert 1 to estimate the calculation?
assert len(base_pred) == val.shape[0]
# Base Pred Normal
base_pred = [0.0] * (val.shape[0])
# Baseline Performance
from sklearn.metrics import accuracy_score
from sklearn.metrics import recall_score
from sklearn.metrics import precision_score
# Evaluate Performance
accuracy = accuracy_score(y_val, base_pred)
precision = precision_score(y_val, base_pred)
recall = recall_score(y_val, base_pred)
print('-- Baseline Model Performance --')
print('Accuracy: ', accuracy)
print('Precision: ', precision)
print('Recall: ', recall)<jupyter_output>-- Baseline Model Performance --
Accuracy:  0.7461112851028291
Precision:  0.0
Recall:  0.0<jupyter_text>Playing Around with Confusion Matrix Metrics ![image.png](attachment:b7057988-68fa-4552-b968-bb0a55b5874d.png)<jupyter_code>from sklearn.metrics import confusion_matrix
cf = confusion_matrix(val['target'], base_pred)
cf
TN = cf[0][0]
FN = cf[1][0]
FP = cf[0][1]
TP = cf[1][1]
# I choose to base prediction correctness on recall as opposed to precision because..
# Assumption: False Negatives are More Costly than False Positives
# ie. A potential job-changer that is missed by the system is more costly
#     than a non-job-changer contacted.
# Precision on Neg?
# Of all Negative Guesses 75% of them were correct.
TN / (TN + FN)
# This is Recall on Neg?
# This means of all of our Actual Negs we Predicted All of them
# Not 1 neg was left behind... this might not be the best performance metric for us
TN / (TN + FP)
# Precision Traditional
# TP / (TP + FP) -> No Precision Since there were 0 Pos Guesses
# Traditional Recall
TP / (TP + FN)<jupyter_output><empty_output><jupyter_text>**Build Predictor Pipeline**<jupyter_code>assert X_val.columns.all() == X_train.columns.all()
# Imports
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OrdinalEncoder
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import recall_score
from sklearn.metrics import precision_score
# -- Tree Based Model --
# Tree based model works better than linear regression.

# T- See if Specifying the Order for Each Col Improves Performance in OrdEncoder. -T

enc = OrdinalEncoder()
std_scaler = StandardScaler()
rf = RandomForestClassifier()

pipe = Pipeline(steps=[
    ('enc', enc), 
    ('std_scaler', std_scaler),
    ('rf', rf)
]
)
model = pipe.fit(X_train, y_train)
pred = model.predict(X_val)
# Evaluate Performance
accuracy = model.score(X_val, y_val)
precision = precision_score(y_val, pred)
recall = recall_score(y_val, pred)
print('-- Tree Based Model Performance --')
print('Accuracy: ', accuracy)
print('Precision: ', precision)
print('Recall: ', recall)
# -- Logistric Regression Attempt --

# T- See if Specifying the Order for Each Col Improves Performance in OrdEncoder. -T
enc = OrdinalEncoder()
std_scaler = StandardScaler()
logistic = LogisticRegression(max_iter=10000)

pipe = Pipeline(steps=[
    ('enc', enc), 
    ('std_scaler', std_scaler),
    ('logistic', logistic)
]
)
model = pipe.fit(X_train, y_train)

pred = model.predict(X_val)

# Evaluate Performance
accuracy = model.score(X_val, y_val)
precision = precision_score(y_val, pred)
recall = recall_score(y_val, pred)
print('-- Logistic Regression Performance --')
print('Accuracy: ', accuracy)
print('Precision: ', precision)
print('Recall: ', recall)<jupyter_output>-- Logistic Regression Performance --
Accuracy:  0.7606221943835474
Precision:  0.5614500442086648
Recall:  0.2611019736842105
{'fraction_non_alphanumeric': 0.09738892398399664, 'fraction_numerical': 0.030322173089071383, 'mean_word_length': 4.4591954022988505, 'pattern_counts': {'":': 0, '<': 30, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '588413', 'license': 'MIT', 'n_tokens_mistral': 3189, 'n_tokens_neox': 2889, 'n_words': 969, 'path': 'model.ipynb', 'repo_name': 'taylor-curran/job-changers', 'size': '9498'}