<jupyter_start><jupyter_text>Create Ideal Profiles for Various Data Science Roles "Am I a better fit for a Data Scientist or a Machine Learning Engineer role?", "Which skills should I highlight on my resume to make my resume more relevant?"...These questions are critical for data science enthusiasts and job seekers. In this project, we'll use data science techniques to analyze data science roles :)   The analysis will be based on the different data science roles as discussed in [this post](https://www.datakademy.com/blog/1850977/navigating-data-science-career-landscape).Let's get started!   temp notesto answer burning questions, such as what big data technology to learn? which role is the best fit based on my skills? We'll approach this project in an iterative way:- Iteration 1: Visualize different DS ideal profiles based on intuition (DONE)- Iteration 2: Scrape data from Indeed to obtain job postings, recreate plots, and validate previous plots based on data obtained- Iteration 3: Use text learning techniques to uncover more insights- Iteration 4: Lable each job posting based on definition made in [this post](https://www.datakademy.com/blog/1850977/navigating-data-science-career-landscape) and build a classfication model- Iteration 5: Deploy the model online using AWS In this project, we'll use spider plots that can be found [here](https://python-graph-gallery.com/392-use-faceting-for-radar-chart/) to visuazlie different data science ideal profiles.In the sample mentioned above, the data frame with below structure is used:<jupyter_code>import pandas as pd

df = pd.DataFrame({
'group': ['A','B','C','D'],
'var1': [38, 1.5, 30, 4],
'var2': [29, 10, 9, 34],
'var3': [8, 39, 23, 24],
'var4': [7, 31, 33, 14],
'var5': [28, 15, 32, 14]
})
df<jupyter_output><empty_output><jupyter_text>However, the csv file I created manually is like this:<jupyter_code># Read in the data manually created
df = pd.read_csv('ideal_profiles.csv')
df<jupyter_output><empty_output><jupyter_text>To make my df compliant, let's use the following transformations:<jupyter_code># We don't need the last column
df = df.iloc[:,:-1]
df
# We need to transpose the df to get the desired format
df = df.T
df
df.reset_index(inplace=True)
df
# Now we need the first row as the header 
df.columns = df.iloc[0]
df
# Delete the first row
df.drop(0, inplace=True)
df
# Reset the index to make sure it'll work properly when plotting below
df.reset_index(inplace=True)
df.drop(columns='index', inplace=True)
df
# Rename the first column
df.rename(columns={'Skill Categories': 'group'}, inplace=True)
df
# Libraries
import matplotlib.pyplot as plt
from math import pi
%matplotlib inline

# Set data
#df = pd.DataFrame({
#'group': ['Data Analyst','Data Scientist','Data Engineer','Machine Learning Engineer'],
#'Business': [50, 60, 10, 10],
#'Database': [30, 50, 60, 40],
#'Data Wrangling': [30, 50, 60, 50],
#'Data Visualization': [50, 60, 20, 40],
#'Math & Stats': [30, 60, 20, 50],
#'ML Modeling': [0, 50, 30, 60],
#'Programming': [20, 40, 60, 60],
#'Big Data': [10, 40, 60, 40],
#'Software Engineering': [0, 30, 60, 60],
#'ML Deployment': [0, 20, 50, 60],
#'Data Engineering': [0, 30, 60, 50]
#})

# ------- PART 1: Define a function that do a plot for one line of the dataset!

def make_spider(df, row, title, color):

    # number of variable
    skill_categories=list(df)[1:]
    N = len(skill_categories)

    # What will be the angle of each axis in the plot? (we divide the plot / number of variable)
    angles = [n / float(N) * 2 * pi for n in range(N)]
    angles += angles[:1]

    # Initialise the spider plot
    ax = plt.subplot(2,2,row+1, polar=True, )

    # If you want the first axis to be on top:
    #ax.set_theta_offset(pi / 2)
    ax.set_theta_direction(-1)

    # Draw one axe per variable + add labels labels yet
    plt.xticks(angles[:-1], skill_categories, color='grey', size=8)

    # Draw ylabels
    ax.set_rlabel_position(0)
    plt.yticks([10,20,30,40,50,60], ["10","20","30","40","50","60"], color="grey", size=7)
    plt.ylim(0,60)

    # Ind1
    values=df.loc[row].drop('group').values.flatten().tolist()
    values += values[:1]
    ax.plot(angles, values, color=color, linewidth=2, linestyle='solid')
    ax.fill(angles, values, color=color, alpha=0.4)

    # Add a title
    plt.title(title, size=11, color=color, y=1.1)

# ------- PART 2: Apply to all individuals
# initialize the figure
my_dpi=96
plt.figure(figsize=(1000/my_dpi, 1000/my_dpi), dpi=my_dpi)

# Create a color palette:
my_palette = plt.cm.get_cmap("Set2", len(df.index))

# Loop to plot
for row in range(0, len(df.index)):
    make_spider(df=df, row=row, title=df['group'][row], color=my_palette(row))

# Adjust space between subplots
plt.subplots_adjust(hspace=0.3)

# Save the plot as a picture
plt.savefig('ideal-profiles.png')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1342821782178218, 'fraction_numerical': 0.04414191419141914, 'mean_word_length': 4.644935972060535, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 3, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '638968', 'license': 'MIT', 'n_tokens_mistral': 1794, 'n_tokens_neox': 1587, 'n_words': 581, 'path': 'ideal_profiles.ipynb', 'repo_name': 'akash8190/Ideal-profile-for-data-scientist-project', 'size': '4848'}