<jupyter_start><jupyter_text>DAY 7 - DATA ANALYST JOB ANALYSIS USING THE DATA ANALYST DATASETThis dataset was created by picklesueat and contains more than 2000 job listing for data analyst positions, with features such as:* Salary Estimate* Location* Company Rating* Job Description  and more.<jupyter_code>import pandas as pd
import numpy as np
df = pd.read_csv(r"C:\Users\seyi\Desktop\MY DATA SCIENCE WORKS, TUTORIAL AND JOURNEY\100 days of DS\Resources\DataAnalyst.csv")
df<jupyter_output><empty_output><jupyter_text>Data Cleaning<jupyter_code>df.isnull()
df.isnull().sum()
df.isnull().any()
# Performing Data Cleaning on the Company Name Column
df['Company Name'] = df['Company Name'].fillna(method = 'bfill')
# Checking if the data still contains null values
df.isnull().sum()
df.head()<jupyter_output><empty_output><jupyter_text>If we notice, we would see that some of the columns have -1 as their values, so we need to replace it with Nan and then clean<jupyter_code>df=df.replace(-1,np.nan)
df=df.replace(-1.0,np.nan)
df=df.replace('-1',np.nan)
df.head()<jupyter_output><empty_output><jupyter_text>We can see that it has been changed, we can now clean<jupyter_code>df.isnull().sum()
df['Company Name'],_=df['Company Name'].str.split('\n', 1).str
df['Job Title'],df['Department']=df['Job Title'].str.split(',', 1).str
df['Salary Estimate'],_=df['Salary Estimate'].str.split('(', 1).str
df.head()<jupyter_output><empty_output><jupyter_text>Splitting the Salary Estimate into max and min salary<jupyter_code>df['Min_Salary'],df['Max_Salary']=df['Salary Estimate'].str.split('-').str
df['Min_Salary']=df['Min_Salary'].str.strip(' ').str.lstrip('$').str.rstrip('K').fillna(0).astype('int')
df['Max_Salary']=df['Max_Salary'].str.strip(' ').str.lstrip('$').str.rstrip('K').fillna(0).astype('int')
df.head()
# Remvoing the Unnamed and Salary Estimate column
df.drop(['Unnamed: 0', 'Salary Estimate'], axis=1 , inplace = True)
df.head()<jupyter_output><empty_output><jupyter_text>Showing the Salary of a Data Analyst (Min Salary)<jupyter_code>df[(df['Job Title'] == 'Data Analyst') & (df['Min_Salary'])]<jupyter_output><empty_output><jupyter_text>Showing the Salary of a Data Analyst (Max Salary)<jupyter_code>df[(df['Job Title'] == 'Data Analyst') & (df['Max_Salary'])]<jupyter_output><empty_output><jupyter_text>Top 20 cities with their minimum and maximum salaries<jupyter_code>df.groupby('Location')[['Max_Salary','Min_Salary']].mean().sort_values(['Max_Salary','Min_Salary'],ascending=False).head(20)<jupyter_output><empty_output><jupyter_text>Top 20 Roles with their minimum and maximum salaries<jupyter_code>df.groupby('Job Title')[['Max_Salary','Min_Salary']].mean().sort_values(['Max_Salary','Min_Salary'],ascending=False).head(20)<jupyter_output><empty_output><jupyter_text>Revenue Generation<jupyter_code>def filter_revenue(x):
    revenue=0
    if(x== 'Unknown / Non-Applicable' or type(x)==float):
        revenue=0
    elif(('million' in x) and ('billion' not in x)):
        maxRev = x.replace('(USD)','').replace("million",'').replace('$','').strip().split('to')
        if('Less than' in maxRev[0]):
            revenue = float(maxRev[0].replace('Less than','').strip())
        else:
            if(len(maxRev)==2):
                revenue = float(maxRev[1])
            elif(len(maxRev)<2):
                revenue = float(maxRev[0])
    elif(('billion'in x)):
        maxRev = x.replace('(USD)','').replace("billion",'').replace('$','').strip().split('to')
        if('+' in maxRev[0]):
            revenue = float(maxRev[0].replace('+','').strip())*1000
        else:
            if(len(maxRev)==2):
                revenue = float(maxRev[1])*1000
            elif(len(maxRev)<2):
                revenue = float(maxRev[0])*1000
    return revenue
        
df['Max_revenue']=df['Revenue'].apply(lambda x: filter_revenue(x))<jupyter_output><empty_output><jupyter_text>Revenue for the different Sectors<jupyter_code>df.groupby('Sector')[['Max_revenue']].mean().sort_values(['Max_revenue'],ascending=False).head(20)<jupyter_output><empty_output><jupyter_text>Revenue for the different Industries<jupyter_code>df.groupby('Industry')[['Max_revenue']].mean().sort_values(['Max_revenue'],ascending=False).head(20)<jupyter_output><empty_output><jupyter_text>The different Job Descriptions asked of for for the following Job titles and the various Comapnies<jupyter_code>df['Job Description'],_=df['Job Description'].str.split('\n', 1).str
print('The following are the job description:'), 
df.groupby('Job Title')[['Job Description', 'Job Title', 'Company Name']].head(30)<jupyter_output>The following are the job description:<jupyter_text>What were the companies established after 2000<jupyter_code>df[(df['Founded'] > 2000) & (df['Company Name'])]<jupyter_output><empty_output><jupyter_text>Looking at the Rating of the different Job Titles<jupyter_code>df.groupby('Rating').max()<jupyter_output><empty_output><jupyter_text>What Jobs are Easy to Apply<jupyter_code>df[(df['Easy Apply'] == 'True')].head(10)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.17326732673267325, 'fraction_numerical': 0.01405940594059406, 'mean_word_length': 6.34156976744186, 'pattern_counts': {'":': 0, '<': 66, '<?xml version=': 0, '>': 65, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '411013', 'license': 'MIT', 'n_tokens_mistral': 1831, 'n_tokens_neox': 1759, 'n_words': 330, 'path': 'Resources/Day 7 - Data Analyst Job Analysis.ipynb', 'repo_name': 'Drimdave/100-Days-of-DS---Going-Pro-In-3-Months', 'size': '5050'}