<jupyter_start><jupyter_text>JOB POSTING KEY WORDS FINDER This program finds the keywords for the job posting.  If you are interested in Industry specific job posting, >  1. Set extract = " ">  2. In the file of Job Description, Enter job description of atleast 10 jobs (the more the better)  If you are interested in Technical or Soft skills key words for the job posting,>  Enter "tech" for Technical and "soft" for Soft skills  • • • Enter Job Title here • • •<jupyter_code>JobTitle = 'Data Scientist'<jupyter_output><empty_output><jupyter_text>• • • Enter the type of skills you want to extract • • •   Enter "tech" for Technical and "soft" for Soft skills here<jupyter_code>extract = "tech"
#extract = "soft"
#extract = " "<jupyter_output><empty_output><jupyter_text>• • • This Section is for the job specific skills to add • • • For example, Here Skills are shown for the IT industry (Specifically, Data Science/Data Analyst positions)<jupyter_code>Popular_Job_Specific_Skills_tech = ['python', 'r', 'nlp', 'nltk' ,'pytorch', 'sql', 'ml', 'al', 'vision', 'cloud', 'scikit-learn', 'Tensorflow', 'software', 'machine', 'statistic', 'mathematics', 'algorithm', 'spark', 'java', 'tableau', 'database', 'hadoop', 'scala', 'hive', 'neural', 'deep', 'javascript', 'aws', 'scikit', 'bi', 'panda', 'excel', 'linux', 'pig', 'etl', 'matlab', 'nosql', 'agile', 's3', 'mysql', 'numpy', 'matplotlib', 'mongodb', 'azure', 'ec2', 'hive']

Popular_Job_Specific_Skills_soft = ['storytelling', 'analysis', 'problem-solving', 'solving', 'thinking', 'initiative', 'detail', 'team', 'smart', 'communication', 'visualization']<jupyter_output><empty_output><jupyter_text>• • • This Section is to scale specific skills from the Job_Specific_Skills parameter • • • Must be integer Try value between 1 to 4. Above it, will provide no proper result<jupyter_code>scaling_parameter = 3<jupyter_output><empty_output><jupyter_text>Install library<jupyter_code>#!pip install NLTK
#!pip install WordCloud<jupyter_output><empty_output><jupyter_text>Importing Required Libraries<jupyter_code>import nltk
import string
import re
from wordcloud import WordCloud
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Read the Job Description<jupyter_code>with open("Job Description.txt", 'r') as jobDes:  
    rawdata = jobDes.read()<jupyter_output><empty_output><jupyter_text>Converting all text to Lower case<jupyter_code>rawdata = rawdata.lower()
#rawdata<jupyter_output><empty_output><jupyter_text>Import Stopwords<jupyter_code>from nltk.corpus import stopwords
stopwords = nltk.corpus.stopwords.words('english')<jupyter_output><empty_output><jupyter_text>Adding Extra words stopwords (General Words, which have high frequency in job postings)<jupyter_code>stopwords = stopwords+["tools","system", "skills", "work", "working", "analytic", "ability", "model", "plus","technologies", "business", "etc", "advanced", "strong","knowledge"]<jupyter_output><empty_output><jupyter_text>Token Generation Function<jupyter_code>def token(rawdata):
    text = re.split('\W+', rawdata)
    return text<jupyter_output><empty_output><jupyter_text>Remove Stopwords Function<jupyter_code>def clean_text(rawdata):
    text = [word for word in rawdata if word not in stopwords]   # Remove stopwords
    return text<jupyter_output><empty_output><jupyter_text>Lemmatizer (Choosing the Root word based on dictionary) Function<jupyter_code>lemmatizer = nltk.WordNetLemmatizer()
def lemmatize_text(text):
    lemmatize_tex = [lemmatizer.lemmatize(word) for word in text]
    return lemmatize_tex<jupyter_output><empty_output><jupyter_text>Stemming (Choosing the simmilar word starting) Function<jupyter_code>ps = nltk.PorterStemmer()
def stemming_text(text):
    stemming_text = [ps.stem(word) for word in text]
    return stemming_text<jupyter_output><empty_output><jupyter_text>Job Specific **TECHNICAL SKILLS** Function<jupyter_code>def tech_skills(data):
    temp_list = []
    global Popular_Job_Specific_Skills_tech
    global scaling_parameter
    for word in rawdata:
        if word in Popular_Job_Specific_Skills_tech:
            #print("====" + word)
            temp_list.append(word)
    temp_list = data + scaling_parameter * temp_list
    return temp_list<jupyter_output><empty_output><jupyter_text>Job Specific **SOFT SKILLS** Function<jupyter_code>def soft_skills(data):
    temp_list = []
    global Popular_Job_Specific_Skills_soft
    global scaling_parameter
    for word in rawdata:
        if word in Popular_Job_Specific_Skills_soft:
            #print("====" + word)
            temp_list.append(word)
    temp_list = data + scaling_parameter * temp_list
    return temp_list<jupyter_output><empty_output><jupyter_text>Token Generate<jupyter_code>rawdata = token(rawdata)
#rawdata<jupyter_output><empty_output><jupyter_text>Extract Tech or Soft Skills, If required<jupyter_code>if (extract == 'soft'):
    rawdata = soft_skills(rawdata)
elif(extract == 'tech'):
    rawdata = tech_skills(rawdata)<jupyter_output><empty_output><jupyter_text>Job title Weightage<jupyter_code>rawdata_title = JobTitle.lower()
rawdata_title = token(rawdata_title)
rawdata_title<jupyter_output><empty_output><jupyter_text>Giving Weightage of 2 words to Job Title<jupyter_code>rawdata = rawdata + rawdata_title
rawdata = rawdata + rawdata_title
import string
string.punctuation
clean_rawdata = clean_text(rawdata)<jupyter_output><empty_output><jupyter_text>Call Stemmer (Not required for DATA or Software related jobs, as most of the words starts with simmilar letters)<jupyter_code># st_data = stemming_text(clean_rawdata)
# rawdata = st_data<jupyter_output><empty_output><jupyter_text>Call Lemmetizer<jupyter_code>lm_data = lemmatize_text(clean_rawdata)
rawdata = lm_data
#rawdata<jupyter_output><empty_output><jupyter_text>To Count the frequency of each word (For Refrence only)<jupyter_code>import pandas as pd
df = pd.DataFrame(rawdata) 
list_of_words = df[0].value_counts()
df = pd.DataFrame(list_of_words)
def print_full(x):
    pd.set_option('display.max_rows', len(x))
    print(x)
    pd.reset_option('display.max_rows')

print_full(df)<jupyter_output>0
data            17
experience       9
management       4
r                4
mathematics      4
python           4
matlab           4
process          3
multiple         3
statistical      2
source           2
using            2
level            2
use              2
practice         2
integrity        2
stakeholder      2
scientist        2
project          2
sa               2
analysis         2
focus            2
governance       2
requirement      2
quality          2
complex          2
science          2
influence        1
warehousing      1
develop          1
case             1
title            1
scienti          1
solve            1
quantitative     1
industry         1
example          1
issue            1
dynamic          1
internet         1
department       1
supply           1
job              1
field            1
identify         1
similar          1
m                1
insight          1
education        1
recommend        1
provide          1
customer    [...]<jupyter_text>Detokenizer (To remove '' '' from the wordcloud plot)<jupyter_code>from nltk.tokenize.treebank import TreebankWordDetokenizer
rawdata = TreebankWordDetokenizer().detokenize(rawdata)<jupyter_output><empty_output><jupyter_text>WordCloud<jupyter_code>wordcloud = WordCloud(stopwords=stopwords, max_words=25, \
                      background_color="white").generate(str(rawdata))

plt.imshow(wordcloud)
plt.axis("off")
plt.figure(figsize=(15,10))
plt.show()<jupyter_output><empty_output><jupyter_text>Still need to remove words? • • • Enter here in the list • • •<jupyter_code>stopwords += ["Enter the list here"] 
wordcloud = WordCloud(stopwords=stopwords, max_words=10, \
                      background_color="white").generate(rawdata)

plt.imshow(wordcloud)
plt.axis("off")
plt.figure(figsize=(15,10))
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10413533834586466, 'fraction_numerical': 0.009523809523809525, 'mean_word_length': 4.4852233676975946, 'pattern_counts': {'":': 0, '<': 104, '<?xml version=': 0, '>': 107, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '26', 'dir': 'jupyter-structured-clean-dedup', 'id': '609839', 'license': 'MIT', 'n_tokens_mistral': 2670, 'n_tokens_neox': 2586, 'n_words': 655, 'path': 'Job KeyWord Finder.ipynb', 'repo_name': 'rahil95shah/Job_Key_Word_Finder', 'size': '7980'}