<jupyter_start><jupyter_text>https://www.accessnow.org/transparency-reporting-index/<jupyter_code>users_by_country_year.reset_index()
sum_court_year.reset_index(inplace=True)
sum_court_year_pivoted = sum_court_year.pivot(index='Country', 
                                              columns = 'year', values= 'Items Requested To Be Removed').reset_index()


sum_court_year_pivoted.isna().sum()
sum_court_year_pivoted = sum_court_year_pivoted.drop([2009, 2010, 2011], axis=1)
sum_court_year_pivoted_reduced = sum_court_year_pivoted.dropna()
sum_court_year_pivoted_reduced.head()
users_by_country_year = users_by_country_year.drop(['1996_y', '1997_y', '1998_y', '1999_y', '2000_y',
                                                    '2001_y', '2002_y', '2003_y', '2004_y', '2005_y', '2006_y', '2007_y', '2008_y', 
                                                    '2009_y', '2010_y', '2011_y'], axis=1)



users_by_country_year.head()
google_requests_users = pd.merge(sum_court_year_pivoted_reduced,
        users_by_country_year,
        left_on="Country",
        right_on="Country Name",
        how="inner")
len(list(sum_court_year_pivoted_reduced.Country.unique()))
##
len(sum_court_year_pivoted_reduced)
len(users_by_country_year)

len(sum_court_year_pivoted_reduced.Country.value_counts())
sum_court_year_pivoted_reduced.columns
google_requests_users_year[:10]
google_requests_users['2015_y']
google_requests_users['transparency_metric'] = (google_requests_users[2012]/google_requests_users['2012_y'] +
    google_requests_users[2013]/google_requests_users['2013_y'] +
    google_requests_users[2014]/google_requests_users['2014_y'] +
    google_requests_users[2015]/google_requests_users['2015_y'] +
    google_requests_users[2016]/google_requests_users['2016_y'] +
    google_requests_users[2017]/google_requests_users['2017_y'])*1000000

google_requests_sorted = google_requests_users.sort_values(by=['transparency_metric'], ascending=False)
google_requests_sorted[google_requests_sorted.Country == 'India']['transparency_metric']
google_requests_sorted[google_requests_sorted.Country == 'United States']['transparency_metric']
google_requests_sorted[google_requests_sorted.Country == 'Russia']['transparency_metric']
google_requests_sorted
ax = google_requests_sorted.plot.line(x='Country', y='transparency_metric', rot=1)<jupyter_output><empty_output><jupyter_text>In an ideal world, we would expect this graph to be normally distributed instead of a long tail<jupyter_code>ax = google_requests_sorted[:10].plot.bar(x='Country', y='transparency_metric')
ax = google_requests_sorted[:20].plot.bar(x='Country', y='transparency_metric')
google_requests_sorted_by_users = google_requests.sort_values(by=['2017_y'], ascending=False)
ax = google_requests_sorted_by_users[:20].plot.bar(x='Country', y='transparency_metric')<jupyter_output><empty_output><jupyter_text>On sorting by internet users, i.e., for the 20 most populated countries, Spain, Turkey, UK & Italy seem to have a higher value of of requests per million internet users<jupyter_code>## POTENTIAL METRIC

google_requests_sorted_by_users['transparency_metric'].std()

## HIGHLY FRAGMENTED<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11755485893416928, 'fraction_numerical': 0.04764890282131661, 'mean_word_length': 6.57957244655582, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '303543', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1173, 'n_tokens_neox': 1040, 'n_words': 137, 'path': 'goverment-interference/analysis/government-interference-by-year.ipynb', 'repo_name': 'neha01mittal/III', 'size': '3190'}