<jupyter_start><jupyter_text>Data Scientist Nanodegree Project: Write A Data Science Blog Post <NAME> In this project, I will go through The CRISP-DM Process for Stack Overflow Data - 2017 Survey.  The CRISP-DM Process (Cross Industry Process for Data Mining)1. Business Understanding2. Data Understanding3. Prepare Data4. Data Modeling5. Evaluate the Results6. Deploy  1. Business UnderstandingQ1: What are the most common Country, EmploymentStatus, FormalEducation, and Professional?Q2: what is the formal education has the highest career satisfaction by professional developer respondents?Q3: Which FormalEducation has the highest CareerSatisfaction?   2. Data Understanding<jupyter_code># Import necessary libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline

# Read the data
df = pd.read_csv('survey-results-public.csv')
schema = pd.read_csv('survey-results-schema.csv')
# Print the first five rows of df
df.head()
# Print number of rows and columns of df
df.shape
# Print 5 number summary of df
df.describe()<jupyter_output><empty_output><jupyter_text>Q1: What are the most common Country, EmploymentStatus, FormalEducation, and Professional? 3. Prepare Data<jupyter_code># Columns name with 0 missing values.
columns_no_nulls = set(df.columns[df.isnull().mean()==0])

# Print columns_no_nulls
columns_no_nulls<jupyter_output><empty_output><jupyter_text>As we see from above, the columns are Country, EmploymentStatus, FormalEducation, and Professional with 0 missing values.<jupyter_code># Value counts of Professional status
prof = df['Professional'].value_counts()

# Print Professional status
prof
# Value counts of Top 10 Countries
country = df['Country'].value_counts().nlargest(10)

# Print Countries
country
# Value counts of EmploymentStatus status
empstat = df['EmploymentStatus'].value_counts()

# Print EmploymentStatus status
empstat
# Value counts of FormalEducation status
formedu = df['FormalEducation'].value_counts()

# Print FormalEducation status
formedu<jupyter_output><empty_output><jupyter_text>4. Data Modeling The Most Respondents by Professional<jupyter_code># Create bar chart of the proportion of individuals in each professional category 
(prof/df.shape[0]).plot(kind="bar");
plt.title("The Proportion of Individuals in each Professional Category");
plt.xlabel('Professional Category')
plt.ylabel('Proportion')<jupyter_output><empty_output><jupyter_text>The Most Respondents by The Country<jupyter_code># Create bar chart of the proportion of individuals in each professional category 
(country/df.shape[0]).plot(kind="bar");
plt.title("The Proportion of Country");
plt.xlabel('Country')
plt.ylabel('Proportion')<jupyter_output><empty_output><jupyter_text>The Most Respondents by Employment Status<jupyter_code># Create bar chart of the proportion of individuals in each professional category 
(empstat/df.shape[0]).plot(kind="bar");
plt.title("The Proportion of Employment Status");
plt.xlabel('Employment Status')
plt.ylabel('Proportion')<jupyter_output><empty_output><jupyter_text>The Most Respondents by Formal Education<jupyter_code># Create bar chart of the proportion of individuals in each professional category 
(formedu/df.shape[0]).plot(kind="bar");
plt.title("The Proportion of Formal Education");
plt.xlabel('Formal Education')
plt.ylabel('Proportion')<jupyter_output><empty_output><jupyter_text>5. Evaluate the ResultsFrom The Bar plots above, most respondents are Professional Developer, from the United States, Employed full-time and have a Bachelor's degree.      Q2: What is the FormalEducation has the highest CareerSatisfaction by Professional Developer respondents? 3. Prepare Data<jupyter_code># Value counts of Career Satisfaction
careersat = df['CareerSatisfaction'].value_counts()

# Print Career Satisfaction
careersat
# Create new dataframe for Professional developer and group by FormalEducation and  average of CareerSatisfaction
top_careersat_edu = pd.DataFrame(df.query("Professional == 'Professional developer'").groupby(['FormalEducation']).mean()['CareerSatisfaction'])
# Display the top_careersat_edu dataset 
top_careersat_edu<jupyter_output><empty_output><jupyter_text>4. Data Modeling The FormalEducation has the highest CareerSatisfaction by Professional Developer respondents<jupyter_code># Create Bar plot for top_careersat_edu
top_careersat_edu.plot(kind = "bar");<jupyter_output><empty_output><jupyter_text>5. Evaluate the ResultsFrom the bar plot above, the FormalEducation has the highest CareerSatisfaction by Professional Developer respondents is **Doctoral degree = 7.62**.  Q3: What is the EmploymentStatus has the highest CareerSatisfaction by Professional Developer respondents? 3. Prepare Data<jupyter_code># Value counts of EmploymentStatus
empstat = df['EmploymentStatus'].value_counts()

# Print EmploymentStatus
empstat
# Create new dataframe for Professional developer and group by EmploymentStatus and  average of CareerSatisfaction
top_careersat_emp = pd.DataFrame(df.query("Professional == 'Professional developer'").groupby(['EmploymentStatus']).mean()['CareerSatisfaction'])
# Display the top_careersat_emp dataset 
top_careersat_emp<jupyter_output><empty_output><jupyter_text>4. Data Modeling The EmploymentStatus has the highest CareerSatisfaction by Professional Developer respondents<jupyter_code># Create Bar plot for top_careersat_emp
top_careersat_emp.plot(kind = "bar");<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08385093167701864, 'fraction_numerical': 0.007489952502740226, 'mean_word_length': 7.423076923076923, 'pattern_counts': {'":': 0, '<': 46, '<?xml version=': 0, '>': 46, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '537356', 'license': 'CNRI-Python', 'n_tokens_mistral': 1663, 'n_tokens_neox': 1521, 'n_words': 507, 'path': 'Stack Overflow Data - 2017 Survey.ipynb', 'repo_name': 'LailaHussain/A-Data-Science-Blog-Post-Stack-Overflow-Data-2017-Survey', 'size': '5490'}