<jupyter_start><jupyter_text>Stackoverflow Survey Data AnalysisIn this notebook, I analyze data for Stackoverflow developer surveys conducted over nine years 2011 - 2019, and answer some questions. Data is available at: https://insights.stackoverflow.com/survey  Business UnderstandingUsing the survey data, I would like to answer the following questions:    1. What is the trend of top Programming Languages over the years    2. What is the trend of job satisfcation over the years    3. What are the Programming Languages gaining/ losing popularity currently    4. What are the Database environments gaining/ losing popularity currently    5. What are the Platforms gaining/ losing popularity currently    6. What are the Web Frameworks gaining/ losing popularity currently  Loading Libraries and Data<jupyter_code>#load in libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import collections
%matplotlib inline
#load 2011 data
df_survey_results_11 = pd.read_csv('./Datasets/Stack_Overflow/2011_Stack_Overflow_Survey_Results.csv', encoding = "ISO-8859-1")
#load 2012 data
df_survey_results_12 = pd.read_csv('./Datasets/Stack_Overflow/2012_Stack_Overflow_Survey_Results.csv', encoding = "ISO-8859-1")
#load 2013 data
df_survey_results_13 = pd.read_csv('./Datasets/Stack_Overflow/2013_Stack_Overflow_Survey_Responses.csv', \
                    encoding = "ISO-8859-1", low_memory=False)
#load 2014 data
df_survey_results_14 = pd.read_csv('./Datasets/Stack_Overflow/2014_Stack_Overflow_Survey_Responses.csv', \
                                   encoding = "ISO-8859-1")
#load 2015 data
df_survey_results_15 = pd.read_csv('./Datasets/Stack_Overflow/2015_Stack_Overflow_Developer_Survey_Responses.csv', \
                                   low_memory=False)
#load 2016 data
df_survey_results_16 = pd.read_csv('./Datasets/Stack_Overflow/2016_Stack_Overflow_Survey_Responses.csv')
#load 2017 data
df_survey_results_17 = pd.read_csv('./Datasets/Stack_Overflow/2017_survey_results_public.csv')
#load 2018 data
df_survey_results_18 = pd.read_csv('./Datasets/Stack_Overflow/2018_survey_results_public.csv', low_memory=False)
#load 2019 data
df_survey_results_19 = pd.read_csv('./Datasets/Stack_Overflow/2019_survey_results_public.csv')<jupyter_output><empty_output><jupyter_text>Functions to help in Analysis<jupyter_code># Some of the earlier datasets have response to a question (e.g. 'Which languages are you proficient in?') spanning across
# many columns - with multiplt Unnamed columns following the column with question. This method concatenates the response 
# across columns in one and drops the Unnamed coumns

# one dataset (2015) contains an empty top row
def clean_unnamed(df):
    '''
    Combine mutiple Unnamed columns corresponding to same question. Also remove any empty top rows in dataframe
    INPUT 
        df - a dataframe to clean with Unnamed columns
        
    OUTPUT
        clean_df - a dataframe with no Unnamed columns
    '''
    
    #Check if 1st row needs to be dropped
    all_unnamed = True
    for col in df.columns[:3]:
        if col.lower().find('unnamed:') != 0:
            all_unnamed = False
    
    if all_unnamed:
        df.columns = df.iloc[0]
        df = df.drop(df.index[0])
        
    i = 0
    cols_to_combine = []
    potential_cols_to_combine = []
    for col in df.columns:
        if col.lower().find('unnamed:') == 0:
            # i == 0 is the first column - ignore it
            if i > 0:
                potential_cols_to_combine.append(col)
        else:
            if len(potential_cols_to_combine) > 1 :
                cols_to_combine.append(potential_cols_to_combine)
            potential_cols_to_combine = [col]
            i += 1
            
    clean_df = df
    for col_set in cols_to_combine:
        i = 0
        for col in col_set:
            clean_df[col] = clean_df[col].fillna('')
            if i == 0:
                temp_col = clean_df[col]
                first_col_name = col
                other_cols = []
            else:
                temp_col = temp_col + ";" + clean_df[col]
                other_cols.append(col)
            i += 1
            
        temp_col = temp_col.str.replace('[;]+', ';')
        temp_col = temp_col.str.replace('^; ', '') 
        temp_col = temp_col.str.replace('; $', '') 
        clean_df[first_col_name] = temp_col
        clean_df = clean_df.drop(other_cols, axis=1)
        
    return clean_df

#Combine columns - for some years data for one question has been collected in multiple columns
#(e.g. year 2015)
def concatenate_cols(df, col_name):
    '''
    Combine mutiple columns corresponding to same question.
    INPUT 
        df - a dataframe with columns to combine
        col_name - partial name of column to combine
        
    OUTPUT
        clean_df - a dataframe with combined column
    '''        
    cols_to_combine = []
    for col in df.columns:
        if col.lower().find(col_name.lower()) == 0:
            cols_to_combine.append(col)
            
    clean_df = df
    i = 0
    for col in cols_to_combine:
        clean_df[col] = clean_df[col].fillna('')
        if i == 0:
            temp_col = clean_df[col]
        else:
            temp_col = temp_col + ";" + clean_df[col]
        i += 1

    temp_col = temp_col.str.replace('[;]+', ';')
    temp_col = temp_col.str.replace('^; ', '') 
    temp_col = temp_col.str.replace('; $', '') 
    clean_df[col_name] = temp_col
    clean_df = clean_df.drop(cols_to_combine, axis=1)
        
    return clean_df
# for year 2014, there is no job satisfaction
def get_common_job_sat_values (col_looking_for_job, col_visit_board, col_career_profile):
    """
    Create job satisfaction values along with percentage of users from three columns - Looking for Job, 
    Visit Job borad frequency, Have a job profile
    Inputs:
        col_looking_for_job - col with information of user looking for job or not
        col_visit_board
    """
    
    all_values = []
    i = 0
    for looking_for_job in col_looking_for_job:
        looking_for_job = looking_for_job.strip().lower()
        visit_board = col_visit_board[i].strip().lower()
        career_profile = col_career_profile[i].strip().lower()
        i += 1
        if looking_for_job == '':
            val = ''
        else:
            val = 'neither satisfied nor dissatisfied'
            if looking_for_job == 'no':
                if visit_board == 'never' and career_profile == 'no thank you':
                    val = 'extremely satisfied'
                elif visit_board == 'never' or career_profile == 'no thank you':
                    val = 'slightly satisfied'
            else:
                if visit_board == 'daily' and career_profile == 'yes':
                    val = 'extremely dissatisfied'
                elif visit_board == 'daily' or career_profile == 'yes':
                    val = 'slightly dissatisfied'
        
        if val != '':
            all_values.append(val)
    
    count_dict = collections.Counter(all_values)
    most_common = count_dict.most_common()
    most_common_as_percent = [(lang, round((count * 100/len(all_values)), 2)) for lang, count in most_common]
    
    return most_common_as_percent

#create data frames for satisfaction
def create_sat_data_frame(percent, year):
    """
    Create data frame from array of tupples and year
    inputs:
        percent - array of tupples with Name and percent values
        year - year of the values
    """
    sat_pd = pd.DataFrame(percent)
    sat_pd = sat_pd.transpose()
    sat_pd.columns = sat_pd.iloc[0]
    sat_pd = sat_pd.drop(sat_pd.index[0])
    sat_pd.insert(0, 'year', [year])
    return sat_pd
# get top values
def get_top_values(col, top_n=10):
    """
    Returns array of tupples for top options and their percentage in a data frame column with multiple options seperated by ';'
    inputs:
        col - column with mutiple selection data.
        top_n - top n values to return along with percentage. Defaults to 10.
    
    """
    # as langauges were merged with framework, technology questiong in some years - ignoring them
    ignore_arr = ['android', 'angularjs', 'cloud (aws, gae, azure, etc.)', 'none', 'others', \
                  'mongodb', 'node.js', 'reactjs', 'redis', 'wordpress', 'arduino / raspberry pi', \
                  'jquery', 'lamp', 'ios', 'sql server', 'response', 'other (please specify)', \
                  "i don't have a job", \
                  'i wish i had a job!']
    
    replace_dict = {'bash':'bash/shell/powershell',
                    'bash/shell':'bash/shell/powershell',
                    'c++11':'c++',
                    'css':'html/css',
                    'html':'html/css',
                    'html5':'html/css',
                    'objective c':'objective-c',
                    'visual basic':'vb',
                    'visual basic 6':'vb',
                    'so happy it hurts':'extremely satisfied',
                    'i enjoy going to work':'slightly satisfied',
                    'it pays the bills':'neither satisfied nor dissatisfied',
                    "i'm not happy in my job":'slightly dissatisfied',
                    'fml':'extremely dissatisfied',
                    'moderately satisfied':'slightly satisfied',
                    'moderately dissatisfied':'slightly dissatisfied',
                    '0.0':'extremely dissatisfied',
                    '1.0':'extremely dissatisfied',
                    '2.0':'slightly dissatisfied',
                    '3.0':'slightly dissatisfied',
                    '4.0':'neither satisfied nor dissatisfied',
                    '5.0':'neither satisfied nor dissatisfied',
                    '6.0':'neither satisfied nor dissatisfied',
                    '7.0':'slightly satisfied',
                    '8.0':'slightly satisfied',
                    '9.0':'extremely satisfied',
                    '10.0':'extremely satisfied',
                    'i hate my job':'extremely dissatisfied',
                    'hate my job':'extremely dissatisfied',
                    'very dissatisfied':'extremely dissatisfied',
                    "i'm somewhat dissatisfied with my job":'slightly dissatisfied',
                    "i'm neither satisfied nor dissatisfied":'neither satisfied nor dissatisfied',
                    "i'm neither satisfied nor dissatisfied with my job":\
                    'neither satisfied nor dissatisfied',
                    'its a paycheck':'neither satisfied nor dissatisfied',
                    "it's a paycheck":'neither satisfied nor dissatisfied',
                    "i'm somewhat satisfied with my job":'slightly satisfied',
                    'i love my job':'extremely satisfied',
                    'love my job':'extremely satisfied',
                    'very satisfied':'extremely satisfied'
                   }
    
    all_values = []
    non_blank_values = 0
    for val in col:
        arr_to_extend = []
        if isinstance(val, float):
            val = str(val)
        for val_split in val.split(';'):
            val_append = val_split.strip().lower()
            if val_append != '' and val_append not in ignore_arr:
                if val_append in replace_dict:
                    val_append = replace_dict[val_append]
                if val_append not in arr_to_extend:
                    arr_to_extend.append(val_append)

        if len(arr_to_extend) > 0:
            non_blank_values += 1
            all_values.extend(arr_to_extend)
            
    count_dict = collections.Counter(all_values)
    most_common = count_dict.most_common(top_n)
    most_common_as_percent = [(lang, round((count * 100)/non_blank_values, 2)) \
                              for lang, count in most_common]
    
    return most_common_as_percent

# get top values
def get_top_values_without_ignore_or_replace(col, top_n=10):
    """
    Returns array of tupples for top options and their percentage in a data frame column with multiple options seperated by ';'
    inputs:
        col - column with mutiple selection data.
        top_n - top n values to return along with percentage. Defaults to 10.
    
    """
    all_values = []
    non_blank_values = 0
    for val in col:
        arr_to_extend = []
        if isinstance(val, float):
            val = str(val)
        for val_split in val.split(';'):
            val_append = val_split.strip()
            if val_append != '':
                if val_append not in arr_to_extend:
                    arr_to_extend.append(val_append)

        if len(arr_to_extend) > 0:
            non_blank_values += 1
            all_values.extend(arr_to_extend)
            
    count_dict = collections.Counter(all_values)
    most_common = count_dict.most_common(top_n)
    most_common_as_percent = [(val, round((count * 100)/non_blank_values, 2)) for val, count in most_common]
    
    return most_common_as_percent
def plot_values(df, col_to_plot, col_to_count, title='', xlabel='', ylabel='', width=6.4, height=4.8):
    """
    Plot a bar chart for unique values in a data frame column.
    inputs:
        df - Data frame
        col_to_plot - Name of data frame column to plot
        col_to_count - Name of column to use for counting
        title - Title of Plot. Defaults to ''
        xlabel - x axis label. Defaults to ''
        ylabel - y axis label. Defaults to ''
        width - width of plot. Defaults to 6.4
        height - height of plot. Defaults to 4.8
    """
    series_to_plot = df.groupby([col_to_plot])[col_to_count].count()*100/df.shape[0]
    series_to_plot.sort_values(inplace=True)
    ax = series_to_plot.plot.barh(figsize=(width, height))
    plt.title(title)
    plt.xlabel(xlabel)
    plt.ylabel(ylabel)
    for i in ax.patches:
        # get_width pulls left or right; get_y pushes up or down
        ax.text(i.get_width()+.3, i.get_y()+.15, str(round(i.get_width(), 2))+'%', fontsize=10, color='dimgrey')
    plt.show()
    
def plot_top_values(top_values, title='', xlabel='', ylabel='', width=6.4, height=4.8):
    """
    Plot a bar chart from an array of tupples.
    inputs:
        top_values - array of tupples to plot
        title - Title of Plot. Defaults to ''
        xlabel - x axis label. Defaults to ''
        ylabel - y axis label. Defaults to ''
        width - width of plot. Defaults to 6.4
        height - height of plot. Defaults to 4.8
    """
    top_pd = pd.DataFrame(top_values)
    top_pd = top_pd.sort_values(by=[1])
    ax = top_pd.plot.barh(x=0, y=1, figsize=(width, height))
    ax.legend().set_visible(False)
    plt.title(title)
    plt.xlabel(xlabel)
    plt.ylabel(ylabel)
    for i in ax.patches:
        # get_width pulls left or right; get_y pushes up or down
        ax.text(i.get_width()+.3, i.get_y()+.15, str(round(i.get_width(), 2))+'%', fontsize=10, color='dimgrey')
    plt.show()<jupyter_output><empty_output><jupyter_text>Data UnderstandingBelow we try to get basic understanding of the data<jupyter_code>df_survey_results_11.head()
df_survey_results_11.describe()
df_survey_results_12.head()
df_survey_results_12.describe()
df_survey_results_13.head()
df_survey_results_13.describe()
df_survey_results_14.head()
df_survey_results_14.describe()
df_survey_results_15.head()
df_survey_results_15.describe()
df_survey_results_16.head()
df_survey_results_16.describe()
df_survey_results_17.head()
df_survey_results_17.describe()
df_survey_results_18.head()
df_survey_results_18.describe()
df_survey_results_19.head()
df_survey_results_19.describe()<jupyter_output><empty_output><jupyter_text>Data CleaningRemoving Unnamed columns and rows.<jupyter_code>df_survey_results_11 = clean_unnamed(df_survey_results_11)
df_survey_results_12 = clean_unnamed(df_survey_results_12)
df_survey_results_13 = clean_unnamed(df_survey_results_13)
df_survey_results_14 = clean_unnamed(df_survey_results_14)
df_survey_results_15 = clean_unnamed(df_survey_results_15)

#Concatenate results from multiple columns in one
df_survey_results_15 = concatenate_cols(df_survey_results_15, 'Current Lang & Tech:')<jupyter_output><empty_output><jupyter_text>Prepare DataRead in the relevant columns required for analysis. Filling NaNs with blanks where requried.<jupyter_code>lang_col_11 = df_survey_results_11['Which languages are you proficient in?']
lang_col_12 = df_survey_results_12['Which languages are you proficient in?']
lang_col_13 = df_survey_results_13['Which of the following languages or technologies have you used significantly in the past year?']
lang_col_14 = df_survey_results_14['Which of the following languages or technologies have you used significantly in the past year?']
lang_col_15 = df_survey_results_15['Current Lang & Tech:']

# Filling NaN with blanks
lang_col_16 = df_survey_results_16['tech_do'].fillna('')
lang_col_17 = df_survey_results_17['HaveWorkedLanguage'].fillna('')
lang_col_18 = df_survey_results_18['LanguageWorkedWith'].fillna('')
lang_col_19 = df_survey_results_19['LanguageWorkedWith'].fillna('')
#Create clean satisfaction cols to work with
sat_col_11 = df_survey_results_11['Please rate your job/career satisfaction'].fillna('')
sat_col_12 = df_survey_results_12['What best describes your career / job satisfaction? '].fillna('')
sat_col_13 = df_survey_results_13['What best describes your career / job satisfaction?'].fillna('')
looking_job_col_14 = df_survey_results_14['Are you currently looking for a job or open to new opportunities?'].fillna('')
visit_job_board_14 = df_survey_results_14['How often do you visit job boards?'].fillna('')
stack_job_profile_14 = df_survey_results_14['Do you have a Stack Overflow Careers 2.0 Profile?'].fillna('')
sat_col_15 = df_survey_results_15['Job Satisfaction'].fillna('')
sat_col_16 = df_survey_results_16['job_satisfaction'].fillna('')
sat_col_17 = df_survey_results_17['JobSatisfaction'].fillna('')
sat_col_18 = df_survey_results_18['JobSatisfaction'].fillna('')
sat_col_19 = df_survey_results_19['JobSat'].fillna('')
lang_worked_with_19 = df_survey_results_19['LanguageWorkedWith']
lang_desired_to_work_19 = df_survey_results_19['LanguageDesireNextYear']
database_worked_with_19 = df_survey_results_19['DatabaseWorkedWith']
database_desired_to_work_19 = df_survey_results_19['DatabaseDesireNextYear']
platform_worked_with_19 = df_survey_results_19['PlatformWorkedWith']
platform_desired_to_work_19 = df_survey_results_19['PlatformDesireNextYear']
webframe_worked_with_19 = df_survey_results_19['WebFrameWorkedWith']
webframe_desired_to_work_19 = df_survey_results_19['WebFrameDesireNextYear']<jupyter_output><empty_output><jupyter_text>Data ModelingBelow we get the top languages across the years along with percentage of users using the language. We get the percentage of users with various levels of job satifaction. We also get top languages, database environments, platforms and web frameworks.<jupyter_code>top_langs_11 = get_top_values(lang_col_11)
top_langs_12 = get_top_values(lang_col_12)
top_langs_13 = get_top_values(lang_col_13)
top_langs_14 = get_top_values(lang_col_14)
top_langs_15 = get_top_values(lang_col_15)
top_langs_16 = get_top_values(lang_col_16)
top_langs_17 = get_top_values(lang_col_17)
top_langs_18 = get_top_values(lang_col_18)
top_langs_19 = get_top_values(lang_col_19)
sat_percent_11 = get_top_values(sat_col_11)
sat_percent_12 = get_top_values(sat_col_12)
sat_percent_13 = get_top_values(sat_col_13)
sat_percent_14 = get_common_job_sat_values(looking_job_col_14, visit_job_board_14, stack_job_profile_14)
sat_percent_15 = get_top_values(sat_col_15)
sat_percent_16 = get_top_values(sat_col_16)
sat_percent_17 = get_top_values(sat_col_17)
sat_percent_18 = get_top_values(sat_col_18)
sat_percent_19 = get_top_values(sat_col_19)
pd_sat_11 = create_sat_data_frame(sat_percent_11, 2011)
pd_sat_12 = create_sat_data_frame(sat_percent_12, 2012)
pd_sat_13 = create_sat_data_frame(sat_percent_13, 2013)
pd_sat_14 = create_sat_data_frame(sat_percent_14, 2014)
pd_sat_15 = create_sat_data_frame(sat_percent_15, 2015)
pd_sat_16 = create_sat_data_frame(sat_percent_16, 2016)
pd_sat_17 = create_sat_data_frame(sat_percent_17, 2017)
pd_sat_18 = create_sat_data_frame(sat_percent_18, 2018)
pd_sat_19 = create_sat_data_frame(sat_percent_19, 2019)
combined_sat_data = pd.concat([pd_sat_11, pd_sat_12, pd_sat_13, pd_sat_14, pd_sat_15, pd_sat_16, pd_sat_17, \
                               pd_sat_18, pd_sat_19], ignore_index=True, sort=False)

cols = ['year', 'extremely dissatisfied', 'slightly dissatisfied', 'neither satisfied nor dissatisfied', \
        'slightly satisfied', 'extremely satisfied']

combined_sat_data = combined_sat_data[cols]
top_lang_worked_with_19 = get_top_values_without_ignore_or_replace(lang_worked_with_19, top_n=1000)
top_lang_desired_to_work_19 = get_top_values_without_ignore_or_replace(lang_desired_to_work_19, top_n=1000)
top_database_worked_with_19 = get_top_values_without_ignore_or_replace(database_worked_with_19, top_n=1000)
top_database_desired_to_work_19 = get_top_values_without_ignore_or_replace(database_desired_to_work_19, top_n=1000)
top_platform_worked_with_19 = get_top_values_without_ignore_or_replace(platform_worked_with_19, top_n=1000)
top_platform_desired_to_work_19 = get_top_values_without_ignore_or_replace(platform_desired_to_work_19, top_n=1000)
top_webframe_worked_with_19 = get_top_values_without_ignore_or_replace(webframe_worked_with_19, top_n=1000)
top_webframe_desired_to_work_19 = get_top_values_without_ignore_or_replace(webframe_desired_to_work_19, top_n=1000)<jupyter_output><empty_output><jupyter_text>Evaluate the ResultsBelow we visualize the results of our analysis. Top Languages over the years<jupyter_code>plot_top_values(top_langs_11, 'Top Langauges of 2011', 'Percentage of Respondents')
plot_top_values(top_langs_12, 'Top Langauges of 2012', 'Percentage of Respondents')
plot_top_values(top_langs_13, 'Top Langauges of 2013', 'Percentage of Respondents')
plot_top_values(top_langs_14, 'Top Langauges of 2014', 'Percentage of Respondents')
plot_top_values(top_langs_15, 'Top Langauges of 2015', 'Percentage of Respondents')
plot_top_values(top_langs_16, 'Top Langauges of 2016', 'Percentage of Respondents')
plot_top_values(top_langs_17, 'Top Langauges of 2017', 'Percentage of Respondents')
plot_top_values(top_langs_18, 'Top Langauges of 2018', 'Percentage of Respondents')
plot_top_values(top_langs_19, 'Top Langauges of 2019', 'Percentage of Respondents')<jupyter_output><empty_output><jupyter_text>An interactive version of above chart is available at: https://shahzeb-akhtar.github.io/Rank_Chart/  Job Satifaction over the years<jupyter_code>ax = combined_sat_data.plot(kind='bar', stacked=True, x='year', figsize=(10,6), legend=False)
handles, labels = ax.get_legend_handles_labels()
ax.legend(reversed(handles), reversed(labels), loc='center left', bbox_to_anchor=(1.0, 0.5))
#plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5))
plt.show()<jupyter_output><empty_output><jupyter_text>Languages Gaining/Losing Popularity<jupyter_code>plot_top_values(top_lang_worked_with_19, 'Languages Worked With - 2019', 'Percentage of Respondents', '', 15, 20)
plot_top_values(top_lang_desired_to_work_19, 'Languages Desired to work on Next Year - 2019', 'Percentage of Respondents', \
                '', 15, 20)<jupyter_output><empty_output><jupyter_text>An interactive version of above chart is available at: https://shahzeb-akhtar.github.io/Rank_Compare/  Database Environments Gaining/ Losing Popularity<jupyter_code>plot_top_values(top_database_worked_with_19, 'Databases Worked With - 2019', 'Percentage of Respondents')
plot_top_values(top_database_desired_to_work_19, 'Databases Desired to work on Next Year - 2019', 'Percentage of Respondents')<jupyter_output><empty_output><jupyter_text>An interactive version of above chart is available at: https://shahzeb-akhtar.github.io/Rank_Compare/database.html  Platforms Gaining/ Losing Popularity<jupyter_code>plot_top_values(top_platform_worked_with_19, 'Platforms Worked With - 2019', 'Percentage of Respondents')
plot_top_values(top_platform_desired_to_work_19, 'Platforms Desired to work on Next Year - 2019', 'Percentage of Respondents')<jupyter_output><empty_output><jupyter_text>An interactive version of above chart is available at: https://shahzeb-akhtar.github.io/Rank_Compare/platform.html  Web Frameworks Gaining/Losing Popularity<jupyter_code>plot_top_values(top_webframe_worked_with_19, 'Webframes Worked With - 2019', 'Percentage of Respondents')
plot_top_values(top_webframe_desired_to_work_19, 'Webframes Desired to work on Next Year - 2019', 'Percentage of Respondents')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09324954388810054, 'fraction_numerical': 0.031380498682343404, 'mean_word_length': 3.5366930292440686, 'pattern_counts': {'":': 6, '<': 49, '<?xml version=': 0, '>': 53, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '51387', 'license': 'MIT', 'n_tokens_mistral': 8532, 'n_tokens_neox': 7746, 'n_words': 1848, 'path': 'Stack_Overflow_Survey_Analysis.ipynb', 'repo_name': 'shahzeb-akhtar/Stackoverflow_Survey_Analysis', 'size': '24665'}