<jupyter_start><jupyter_text>**Dataset: HR Analytics: Job Change of Data Scientists**  **1.** **Introduction**  **1.1 Team***   <NAME>*   <NAME>*   <NAME>  **1.2 Main question:**A company which is active in Big Data and Data Science wants to hire data scientists among people who successfully pass some courses which conduct by the company. Many people signup for their training. Company wants to know which of these candidates are really wants to work for the company after training or looking for a new employment because it helps to reduce the cost and time as well as the quality of training or planning the courses and categorization of candidates. Information related to demographics, education, experience are in hands from candidates signup and enrollment.Which factors that lead a person to leave current job and change their job to data scientist?* Audience: Jobseeker who want to change job to a data scientist and recruiter of the company who want to look for ideal candidates **Sub questions - Part 1:**__Data Cleaning:__- Is there any duplication in your data?- Is there missing data?- Is there any mislabeled data/errors?- Is there any column that need reformatting for better analysis?__Exploratory Data Analysis:__- For numerical data: How is the data distributed? How are they correlated? Provide summary statistics of these data? Identify outliers, check if they are errors or simply abnormalities in the data. ?- For categorical data: How many categories are there? Are there any difference between those categories? **Sub questions - Part 2:**How many training hours do a job seeker needs to be ready for a job change in data science?Does any of the following factors affect someone's intention of changing their job to data science?*   Gender*   Relevant experience*   Enrolled university*   Education level*   Major*   Recent job*   Company size*   Company type*   City development index<jupyter_code>import pandas as pd 
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import scipy
 hr = pd.read_csv('https://raw.githubusercontent.com/trannguyen135/trannguyen135/main/aug_train.csv')<jupyter_output><empty_output><jupyter_text>**1.3** **Data overview**<jupyter_code>hr.head()
hr.info()
hr.describe()
hr.describe(include=object)<jupyter_output><empty_output><jupyter_text>**Comments:**Problems of dataset:*    Missing values in columns: gender, enrolled_university, education_level, major_discipline, experience, company_size, company_type, last_new_job*    Mislabeled data/errors: relevent_experience, city, company size<jupyter_code>numbers = [f for f in hr.columns if hr.dtypes[f] != 'object']
list_nums = ', '.join(numbers)
list_nums
objects = [f for f in hr.columns if hr.dtypes[f] =='object']
list_objects = ', '.join(objects)
list_objects
# Categorical:
i = 1
for obj in objects:
    print(i, "/", obj, "\t", len(hr[obj].unique()), ":", hr[obj].unique())
    i = i+1
i = 1
for obj in numbers:
    print(i, "/", obj, len(hr[obj].unique()), ":", hr[obj].unique() if len(hr[obj].unique())<10 else '')
    i = i+1<jupyter_output>1 / enrollee_id 19158 : 
2 / city_development_index 93 : 
3 / training_hours 241 : 
4 / target 2 : [1. 0.]<jupyter_text>**2.1** **Data Cleaning - mislabeled data/errors**<jupyter_code>hr.head()
#Rename some columns
hr = hr.rename(columns = ({'city':'city_code', 'experience':'work_experience', 'last_new_job':'most_recent_job'}))
#Replace some values in the work_experience, relevent_experience and most_recent_job column
hr['work_experience'].replace({np.NaN:0,'>20':21,'<1':0},inplace=True)

hr['most_recent_job'].replace({np.NaN:0,'>4':5,'never':0},inplace=True)

hr["relevent_experience"].replace({"Has relevent experience":"yes","No relevent experience":"no"},inplace=True)
#change type of columns
hr['work_experience'] = hr['work_experience'].astype(int)

hr['most_recent_job'] = hr['most_recent_job'].astype(int)
#Get code city to fix data in city column
def get_code_city(city):
    return city.split('_')[1]

#Fix data inn the company_size column
def fix_company_size(x):
    
    if x == "<10":
        return "Local"
    elif x == "50-99" or x == "10/49":
        return "Small"
    elif x == "100-500":
        return "Medium"
    elif x == "500-999":
        return "Upper"
    elif x == "1000-4999" or x == "5000-9999":
        return "Extended"
    elif x == "10000+":
        return "Large"
#Fix data inn the most_recent_job column
def fix_most_recent_job(x):
    if 0 < x <= 1:
        return "0-1"
    elif 1 < x <= 3:
        return "2-3"
    elif 3 < x <= 5:
        return "4-5"
    elif x > 5:
        return "5+"
    else:
        pass
#Fix data in the work_experience column
def fix_work_experience(x):
    if x == 0:
        return "0"
    elif 0 < x <= 3:
        return "1-3"
    elif 3 < x <= 7:
        return "4-7"
    elif 7 < x <= 15:
        return "7-15"
    elif x > 15:
        return "15+"
#Create city_code and apply to get_code_city
hr['city_code'] = hr['city_code'].apply(get_code_city)
#Fix name in company_size column
hr["company_size"] = hr["company_size"].apply(lambda x: fix_company_size(x))
#Fix values in  work_experience column 
hr["work_experience"] = hr["work_experience"].apply(lambda x: fix_work_experience(x))
#Fix values in most_recent_job column 
hr["most_recent_job"] = hr["most_recent_job"].apply(lambda x: fix_most_recent_job(x))
hr.head()
hr.duplicated().sum()<jupyter_output><empty_output><jupyter_text>**2.1** **Data Cleaning - Duplication**<jupyter_code># Display dataframe to look for columns cannot be duplicated
hr<jupyter_output><empty_output><jupyter_text>**Comment:** enrollee_id cannot be duplicated<jupyter_code># Check the overall duplication
hr.duplicated().sum()
# Look for enrollee_id duplicated values
hr['enrollee_id'].duplicated().sum()<jupyter_output><empty_output><jupyter_text>**Commet:** No duplication  **2.2** **Data Cleaning - Missing values**<jupyter_code># Check for NULL values
hr.isnull().sum()<jupyter_output><empty_output><jupyter_text>**Comment:** Null values in columns: 'gender', 'enrolled_university',    'education_level', 'major_discipline', 'comapny_size', 'company_type', 'most_recent_job'. 1. **Gender**<jupyter_code># check for all unique variables
hr['gender'].unique()
# Display the dataframe to look for gender indications 
hr<jupyter_output><empty_output><jupyter_text>**Comment:** No clear indication but assume 'major_discipline' has<jupyter_code># Check relationship with major discipline
hr.groupby('gender')['major_discipline'].value_counts()<jupyter_output><empty_output><jupyter_text>**Notice:** a significant portion of 'Male' is 'STEM' -> replace NULL values have 'major_discipline' as 'STEM' WITH 'Male'<jupyter_code>mask = (hr['major_discipline'] == 'STEM') & (hr['gender'].isnull())
hr.loc[mask, 'gender'] = hr.loc[mask, 'gender'].fillna('Male')
# Check for update
hr.groupby('gender')['major_discipline'].value_counts()
# Check the percentage of remaining NULL values
hr['gender'].isnull().sum() / hr['gender'].count() * 100<jupyter_output><empty_output><jupyter_text>**Comment:** 6.7% is a small percentage -> replace null values with mode value of 'gender<jupyter_code>hr['gender'] = hr['gender'].fillna(hr.gender.mode()[0])
# Final check for gender
hr['gender'].unique()<jupyter_output><empty_output><jupyter_text>2. **enrolled_university**<jupyter_code># Check all unique varialbes
hr['enrolled_university'].unique()
# Display the dataframe to look for indications
hr<jupyter_output><empty_output><jupyter_text>**Comment:** Education level looks like a potential indication<jupyter_code># Check relationship

hr.groupby('enrolled_university')['education_level'].value_counts()<jupyter_output><empty_output><jupyter_text>**Comment:** 'Education_level', actually, shows no indication for 'enrolled_university' and versus.<jupyter_code># Check the percentage of NULL values
hr['enrolled_university'].isnull().sum() / hr['enrolled_university'].count() * 100<jupyter_output><empty_output><jupyter_text>**Comment:** 2.06% is a small percentage -> replace NULL values with mode value of 'enrolled_university'.<jupyter_code>hr['enrolled_university'] = hr['enrolled_university'].fillna(hr.enrolled_university.mode()[0])
# Final check for enrolled_university
hr['enrolled_university'].unique()<jupyter_output><empty_output><jupyter_text>3. **education_level**<jupyter_code># Check all unique variables
hr['education_level'].unique()
# Display the dataframe to look for indications
hr<jupyter_output><empty_output><jupyter_text>**Comment:** Potentially has relationship with 'enrolled_university' but shows in indication as proved above.<jupyter_code># Check the percentage of NULL values
hr['education_level'].isnull().sum() / hr['education_level'].count() * 100<jupyter_output><empty_output><jupyter_text>**Comment:** 2.46% is a small percentage -> replace NULL values with mode value of 'education_level'.<jupyter_code>hr['education_level'] = hr['education_level'].fillna(hr.education_level.mode()[0])
# Final check for education_level
hr['education_level'].unique()<jupyter_output><empty_output><jupyter_text>4. **major_discipline**<jupyter_code># Check for all unique variables
hr['major_discipline'].unique()
# Display dataframe to look for indications
hr<jupyter_output><empty_output><jupyter_text>**Comment:** No clear indication but as proved above, 'Male's tend to be 'STEM' -> replace NULL values have gender as 'Male' with 'STEM'<jupyter_code>mask = (hr['major_discipline'].isnull()) & (hr['gender'] == 'Male')
hr.loc[mask, 'major_discipline'].fillna('STEM', inplace = True)
# Check the percentage of the remaining NULL values
hr['major_discipline'].isnull().sum() / hr['major_discipline'].count() * 100<jupyter_output><empty_output><jupyter_text>**Comment:** 17.21% is still a significant portion -> replace with 'Unknown'<jupyter_code>hr['major_discipline'] = hr['major_discipline'].fillna('Unknown')
# Final check for major_discipline
hr['major_discipline'].unique()<jupyter_output><empty_output><jupyter_text>5. **company_size**<jupyter_code># Check for all unique variables
hr['company_size'].unique()
# Display dataframe to look for indications
hr<jupyter_output><empty_output><jupyter_text>**Comment:** No clear indication but have an assumption that all startups are 'Small'<jupyter_code># Check assumption
hr.groupby('company_type')['company_size'].value_counts()<jupyter_output><empty_output><jupyter_text>**Comment:** No clear indication from 'company_type' for 'company_size' and versus<jupyter_code># Calculate the percentage of NULL values
hr['company_size'].isnull().sum() / hr['company_size'].count() * 100<jupyter_output><empty_output><jupyter_text>**Comment:** 50% is a very huge percentage -> replace NULL values with 'Unknown'<jupyter_code>hr['company_size'] = hr['company_size'].fillna('Unknown')
# Final check for company_size
hr['company_size'].unique()<jupyter_output><empty_output><jupyter_text>6. Company_type<jupyter_code># Check all unique variables
hr['company_type'].unique()<jupyter_output><empty_output><jupyter_text>**Comment:** Only has connection with 'company_size' but as proved above, no clear indication -> replace with 'Unknown'<jupyter_code># Check the percenatge of NULL values
hr['company_type'].isnull().sum() / hr['company_type'].count() * 100<jupyter_output><empty_output><jupyter_text>**Comment:** 47.17% is a huge percentage -> replace NULL values with 'Unknown'<jupyter_code>hr['company_type'] = hr['company_type'].fillna('Unknown')
# Final checl for company_type
hr['company_type'].unique()<jupyter_output><empty_output><jupyter_text>* **Among columns, 'company_size' and 'company_type' have a significant portion of 'Unknown' values**  **3.1** **EDA - For continuous variables** *city_development_index*<jupyter_code>hr['city_development_index'].describe()
hr_median = hr['city_development_index'].median()
hr_median
hr_mode = hr['city_development_index'].mode()
hr_mode
# Visualize data
plt.figure(figsize=(10,5))
plt.subplot(1,2,1)
sns.distplot(hr['city_development_index'])
plt.subplot(1,2,2)
plt.hist(hr['city_development_index'])
plt.show()<jupyter_output>/usr/local/lib/python3.7/dist-packages/seaborn/distributions.py:2557: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).
  warnings.warn(msg, FutureWarning)<jupyter_text>**Comments:** 1.   Not equally distributed2.   Mostly in 0.923.   Negative skewness4.   Has an odd local peak at around 0.6 *training_hours*<jupyter_code>hr['training_hours'].describe()
hr_mode = hr['training_hours'].mode()
hr_mode
plt.figure(figsize=(10,5))
plt.subplot(1,2,1)
sns.distplot(hr['training_hours'])
plt.subplot(1,2,2)
plt.hist(hr['training_hours'])
plt.show()<jupyter_output>/usr/local/lib/python3.7/dist-packages/seaborn/distributions.py:2557: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).
  warnings.warn(msg, FutureWarning)<jupyter_text>**Comments:** 1.   Range is from 0 to 3362.   Not equally distributed3.   Mostly have training hours less than a 100.4.   Positive skewness.<jupyter_code>hr[['city_development_index','training_hours','target']].corr()
# Continuous & continuous
sns.pairplot(hr[['city_development_index','training_hours','target']])<jupyter_output><empty_output><jupyter_text>**Comment:** *   No correlation between 'city_development_index' and 'training_hours'*   'city_development_index' and 'target' have negative correlation*   No correlation between 'training_hours' and 'target'<jupyter_code># boxplot of city_development_index & training_hours
plt.figure(figsize=(8,6))
plt.subplot(121)
plt.title('city_development_index')
plt.boxplot(hr['city_development_index'])
plt.subplot(122)
plt.title('training_hours')
plt.boxplot(hr['training_hours'])
plt.show()<jupyter_output><empty_output><jupyter_text>**Comments:***   For variable 'city_development_index': only one outliner at the lower whisker*   For variable 'training_hour': many ouliners at the upper whisker *Checking outliners of 'city_development_index'*<jupyter_code>Q1 = np.percentile(hr.city_development_index, 25)
Q1
Q3 = np.percentile(hr.city_development_index, 75)
Q3
city_development_index_iqr = Q3 - Q1
city_development_index_iqr 
outliner_ratio_1 = len(hr[(hr['city_development_index'] < (Q1 - 1.5*city_development_index_iqr))])/len(hr['city_development_index'])
outliner_percentage_1 = "{:.2%}".format(outliner_ratio_1)
print('Percentage of outliners:',outliner_percentage_1)
hr_new_1 = hr[(hr['city_development_index'] >= (Q1 - 1.5*city_development_index_iqr))]
plt.boxplot(hr_new_1['city_development_index'])
plt.show()
hr_new_1.city_development_index.describe()
hr.city_development_index.describe()<jupyter_output><empty_output><jupyter_text>**Comments:***    Percentage of outliners is not significant*    Outliners don't create statistically significant difference**-> Consider not to remove outliners** *Checking outliners of 'training_hours'*<jupyter_code>Q1 = np.percentile(hr.training_hours, 25)
Q1
Q3 = np.percentile(hr.training_hours, 75)
Q3
training_hours_iqr = Q3 - Q1
training_hours_iqr 
outliner_ratio = len(hr[(hr['training_hours'] > (Q3 + 1.5*training_hours_iqr ))])/len(hr['training_hours'])
outliner_percentage = "{:.2%}".format(outliner_ratio)
print('Percentage of outliners:',outliner_percentage)
hr_new = hr[(hr['training_hours'] <= (Q3 + 1.5*training_hours_iqr ))]
plt.boxplot(hr_new['training_hours'])
plt.show()
hr_new.training_hours.describe()
hr.training_hours.describe()<jupyter_output><empty_output><jupyter_text>**Comments:***    Percentage of outliners is reasonable*    Outliners don't create statistically significant difference*    Losing insights from abnormal groups in dataset**-> Consider not to remove outliners** *Checking abnormal groups of 'training_hours'*<jupyter_code>training_hours_abnormal = hr[(hr['training_hours'] > (Q3 + 1.5*training_hours_iqr ))]
training_hours_abnormal.head(10)
training_hours_abnormal.describe()
training_hours_abnormal.describe(include=object)<jupyter_output><empty_output><jupyter_text>**Comments:***    Mean of dataset is 248 hours*    Compare to normal dataset, the only different variable is 'work_experience' (work experience in abnormal dataset is shorter(7 - 15 years))**New question: Is there correlation between work experience and training hours?**<jupyter_code>work_train = hr.groupby('work_experience')['training_hours'].mean().sort_values(ascending=True)
work_train<jupyter_output><empty_output><jupyter_text>**Comments:***    Freshers and Juniors (0 - 3 years) have the least training hours*    Seniors (4 - 15 years) have the most training hours **3.2** **EDA - For categorical variables**<jupyter_code>city_count = (hr.groupby('city_code').count())['enrollee_id'].sort_values(ascending=False)
top_10_city = city_count.head(10)
gender_count = (hr.groupby('gender').count())['enrollee_id'].sort_values(ascending=False)
relevant_experience_count = (hr.groupby('relevent_experience').count())['enrollee_id'].sort_values(ascending=False)
university_count = (hr.groupby('enrolled_university').count())['enrollee_id'].sort_values(ascending=False)
education_count = (hr.groupby('education_level').count())['enrollee_id'].sort_values(ascending=False)
major_count = (hr.groupby('major_discipline').count())['enrollee_id'].sort_values(ascending=False)
experience_count = (hr.groupby('work_experience').count())['enrollee_id'].sort_values(ascending=False)
company_size_count = (hr.groupby('company_size').count())['enrollee_id'].sort_values(ascending=False)
company_type_count = (hr.groupby('company_type').count())['enrollee_id'].sort_values(ascending=False)
last_new_job_count = (hr.groupby('most_recent_job').count())['enrollee_id'].sort_values(ascending=False)
print(top_10_city)
print('-'*50)
print(gender_count)
print('-'*50)
print(relevant_experience_count)
print('-'*50)
print(university_count)
print('-'*50)
print(education_count)
print('-'*50)
print(major_count)
print('-'*50)
print(experience_count)
print('-'*50)
print(company_size_count)
print('-'*50)
print(company_type_count)
print('-'*50)
print(last_new_job_count)

plt.figure(figsize=(50,40))
plt.subplots_adjust(bottom=0.3, top=0.7, hspace=0.6)
plt.rc('xtick',labelsize=20)
plt.rc('ytick',labelsize=20)
plt.subplot(251)
sns.set(style="darkgrid")
top_10_city.plot.bar()
plt.xlabel('City code',fontsize=20)
plt.ylabel('Count',fontsize=20)
plt.rc('xtick',labelsize=20)
plt.rc('ytick',labelsize=20)
plt.subplot(252)
sns.set()
gender_count.plot.bar()
plt.xlabel('Gender',fontsize=20)
plt.ylabel('Count',fontsize=20)
plt.rc('xtick',labelsize=20)
plt.rc('ytick',labelsize=20)
plt.subplot(253)
sns.set()
relevant_experience_count.plot.bar()
plt.xlabel('Relevant Experience',fontsize=20)
plt.ylabel('Count',fontsize=20)
plt.rc('xtick',labelsize=20)
plt.rc('ytick',labelsize=20)
plt.subplot(254)
sns.set()
university_count.plot.bar()
plt.xlabel('Enrolled University',fontsize=20)
plt.ylabel('Count',fontsize=20)
plt.rc('xtick',labelsize=20)
plt.rc('ytick',labelsize=20)
plt.subplot(255)
sns.set()
education_count.plot.bar()
plt.xlabel('Education Level',fontsize=20)
plt.ylabel('Count',fontsize=20)
plt.rc('xtick',labelsize=20)
plt.rc('ytick',labelsize=20)
plt.subplot(256)
sns.set()
major_count.plot.bar()
plt.xlabel('Major discippline',fontsize=20)
plt.ylabel('Count',fontsize=20)
plt.rc('xtick',labelsize=20)
plt.rc('ytick',labelsize=20)
plt.subplot(257)
sns.set()
experience_count.plot.bar()
plt.xlabel('Work Experience',fontsize=20)
plt.ylabel('Count',fontsize=20)
plt.rc('xtick',labelsize=20)
plt.rc('ytick',labelsize=20)
plt.subplot(258)
sns.set()
company_size_count.plot.bar()
plt.xlabel('Company Size',fontsize=20)
plt.ylabel('Count',fontsize=20)
plt.rc('xtick',labelsize=20)
plt.rc('ytick',labelsize=20)
plt.subplot(259)
sns.set()
company_type_count.plot.bar()
plt.xlabel('Company type',fontsize=20)
plt.ylabel('Count',fontsize=20)
plt.rc('xtick',labelsize=20)
plt.rc('ytick',labelsize=20)
plt.subplot(2,5,10)
sns.set()
last_new_job_count.plot.bar()
plt.xlabel('Most recent job',fontsize=20)
plt.ylabel('Count',fontsize=20)
plt.show()<jupyter_output><empty_output><jupyter_text>*Analyze `target` variable*<jupyter_code>target_count = hr.groupby('target').count()['enrollee_id']
target_count
sns.set(style="darkgrid")
plt.figure(figsize=(8,5))
ax = sns.countplot(x="target", data=hr)
for p in ax.patches:
    percentage = '{:.1f}%'.format(100 * p.get_height()/len(hr.target))
    x = p.get_x() + p.get_width()/2
    y = p.get_height()*1.05
    ax.annotate(percentage, (x, y),ha='center')
plt.ylim(0,16000)
plt.show()<jupyter_output><empty_output><jupyter_text>**Comments:** Imbalanced dataset, >75% data is 'target 0' (people who are not looking for a job change to data scientist)--- *Analyze categorical variables and 'target'* *Analyze 'gender' and 'target'*<jupyter_code>plt.figure(figsize=(8,6))
sns.countplot(hr["gender"], hue = "target", data = hr)
plt.show()
target_gender = hr.groupby(['gender'])['target'].value_counts(normalize=True).unstack()
target_gender
plt.rcParams["figure.figsize"] = [8, 6]
target_gender.plot.bar(stacked=True)
plt.show()<jupyter_output><empty_output><jupyter_text>**Comments:***    Percentage of female want to change their job is more than percentage of male *Analyze 'relevent_experience' and 'target'*<jupyter_code>plt.figure(figsize=(8,6))
sns.countplot(hr["relevent_experience"], hue = "target", data = hr)
plt.show()
relevant_experience_target = hr.groupby(['relevent_experience'])['target'].value_counts(normalize=True).unstack()
relevant_experience_target
plt.rcParams["figure.figsize"] = [8, 6]
relevant_experience_target.plot.bar(stacked=True)
plt.show()<jupyter_output><empty_output><jupyter_text>**Comments:***    People who have no relevant experience want to change job to data science more than people who have relevant experience *Analyze 'enrolled_university' and 'target'*<jupyter_code>plt.figure(figsize=(8,6))
sns.countplot(hr["enrolled_university"], hue = "target", data = hr)
plt.show()
enrolled_university_target = hr.groupby(['enrolled_university'])['target'].value_counts(normalize=True).unstack()
enrolled_university_target
plt.rcParams["figure.figsize"] = [8, 6]
enrolled_university_target.plot.bar(stacked=True)
plt.show()<jupyter_output><empty_output><jupyter_text>**Comments:***    People who enroll for full time course in university is more motivated to change their job to data science than people who don't enroll to any courses or just do part time course *Analyze 'education_level' and 'target'*<jupyter_code>plt.figure(figsize=(8,6))
sns.countplot(hr["education_level"], hue = "target", data = hr)
plt.show()
education_level_target = hr.groupby(['education_level'])['target'].value_counts(normalize=True).unstack()
education_level_target
plt.rcParams["figure.figsize"] = [8, 6]
education_level_target.plot.bar(stacked=True)
plt.show()<jupyter_output><empty_output><jupyter_text>**Comments:***    People who finished bachelor are looking for a job change to data science more than others *Analyze 'major_discipline' and 'target'*<jupyter_code>plt.figure(figsize=(10,6))
sns.countplot(hr["major_discipline"], hue = "target", data = hr)
plt.show()
major_discipline_target = hr.groupby(['major_discipline'])['target'].value_counts(normalize=True).unstack()
major_discipline_target
plt.rcParams["figure.figsize"] = [8, 6]
major_discipline_target.plot.bar(stacked=True)
plt.show()<jupyter_output><empty_output><jupyter_text>**Comments:***    People who studied STEM and Business are looking for a job change to data science more than others *Analyze 'work_experience' and 'target'*<jupyter_code>plt.figure(figsize=(10,6))
sns.countplot(hr["work_experience"], hue = "target", data = hr)
plt.show()
work_experience_target = hr.groupby(['work_experience'])['target'].value_counts(normalize=True).unstack()
work_experience_target
plt.rcParams["figure.figsize"] = [8, 6]
work_experience_target.plot.bar(stacked=True)
plt.show()<jupyter_output><empty_output><jupyter_text>**Comments:***    People who have limited working experience (0 - 3 years) are looking for a job change to data science more than others *Analyze 'company_size' and 'target'*<jupyter_code>plt.figure(figsize=(10,6))
sns.countplot(hr["company_size"], hue = "target", data = hr)
plt.show()
company_size_target = hr.groupby(['company_size'])['target'].value_counts(normalize=True).unstack()
company_size_target
plt.rcParams["figure.figsize"] = [8, 6]
company_size_target.plot.bar(stacked=True)
plt.show()<jupyter_output><empty_output><jupyter_text>*Analyze 'company_type' and 'target'*<jupyter_code>plt.figure(figsize=(15,6))
sns.countplot(hr["company_type"], hue = "target", data = hr)
plt.show()
company_type_target = hr.groupby(['company_type'])['target'].value_counts(normalize=True).unstack()
company_type_target
plt.rcParams["figure.figsize"] = [8, 6]
company_type_target.plot.bar(stacked=True)
plt.show()<jupyter_output><empty_output><jupyter_text>**Comments:***    People who worked in public sector and early stage startup are looking for a job change to data science more than others *Analyze 'last_new_job' and 'target'*<jupyter_code>plt.figure(figsize=(10,6))
sns.countplot(hr["most_recent_job"], hue = "target", data = hr)
plt.show()
most_recent_job_target = hr.groupby(['most_recent_job'])['target'].value_counts(normalize=True).unstack()
most_recent_job_target
plt.rcParams["figure.figsize"] = [8, 6]
most_recent_job_target.plot.bar(stacked=True)
plt.show()<jupyter_output><empty_output><jupyter_text>**Comments:***    People who spent a short time (0 - 1 year) in their most recent job are looking for a job change to data science more than others *Analyze 'training_hours' and 'target'*<jupyter_code>training_hours_target = hr.groupby('target')['training_hours'].mean()
training_hours_target
plt.rcParams["figure.figsize"] = [8, 6]
training_hours_target.plot.bar()
plt.show()<jupyter_output><empty_output><jupyter_text>**Comments:***    People who are looking for job change to data science spent average 63 training hours *Analyze 'city_development_index' and 'target'*<jupyter_code>city_development_target = hr.groupby('target')['city_development_index'].mean()
city_development_target
plt.rcParams["figure.figsize"] = [8, 6]
city_development_target.plot.bar()
plt.show()<jupyter_output><empty_output><jupyter_text>**Comments:***    People who are looking for job change to data science come from city with lower development index (0.75)  **EXTRA** **Data Standardization**<jupyter_code>hr['training_hours_log'] = np.log(hr['training_hours'])
hr.head()
plt.figure(figsize=(10,5))
plt.subplot(1,2,1)
sns.distplot(hr['training_hours_log'])
plt.subplot(1,2,2)
plt.hist(hr['training_hours_log'])
plt.show()
# Use RobustScaler for variable "city_development_index"
# CODE HERE
# Import sklearn<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14524046801756912, 'fraction_numerical': 0.019414608939578488, 'mean_word_length': 7.582198289515363, 'pattern_counts': {'":': 10, '<': 218, '<?xml version=': 0, '>': 220, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '50', 'dir': 'jupyter-structured-clean-dedup', 'id': '150976', 'license': 'MIT', 'n_tokens_mistral': 9683, 'n_tokens_neox': 8986, 'n_words': 1990, 'path': 'HRanalytics.ipynb', 'repo_name': 'trannguyen135/HRanalytic', 'size': '27120'}