<jupyter_start><jupyter_text>Demographic SharesAdds percentages to the demographics data<jupyter_code>from pandas import DataFrame, read_csv
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np<jupyter_output><empty_output><jupyter_text>We read the data from the csv file, replace the asterisk (protecting values between 1 and 4) with a 2 and add the column 'Total' which we need to calculate the share of each demographic group.<jupyter_code>file = r'../data/unhcr_popstats_export_demographics_all_data.csv'
df = pd.read_csv(file, header=3)
df = df.replace(to_replace='*', value='2')
df.iloc[:, 3:19] = df.iloc[:, 3:19].apply(pd.to_numeric)
df['Total'] = df['F: Total'] + df['M: Total']
df['Total'] = pd.to_numeric(df['Total']).astype(np.int64)
df.dtypes<jupyter_output><empty_output><jupyter_text>We calculate the share of each demographic group and add a column for it (`_share`).<jupyter_code>def calculate_share(demographic):
    df[demographic + '_share'] = df[demographic] / df['Total']
for column in df.iloc[:, 3:19]:
    calculate_share(column)
print(df)<jupyter_output>Year Country / territory of asylum/residence  \
0      2001                             Afghanistan   
1      2001                             Afghanistan   
2      2001                             Afghanistan   
3      2001                                  Angola   
4      2001                                  Angola   
5      2001                                  Angola   
6      2001                                  Angola   
7      2001                                  Angola   
8      2001                                  Angola   
9      2001                                  Angola   
10     2001                                  Angola   
11     2001                                  Angola   
12     2001                                 Albania   
13     2001                                 Albania   
14     2001                                 Albania   
15     2001                                 Albania   
16     2001                                 Albania   
17     200[...]<jupyter_text>We save the dataframe as a csv file.<jupyter_code>df.to_csv('../data/unhcr_demographics_share.csv', encoding='utf-8', index=False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07668576465403261, 'fraction_numerical': 0.0502423975319524, 'mean_word_length': 1.5278396436525612, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 16, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 21, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '503199', 'license': 'MIT', 'n_tokens_mistral': 706, 'n_tokens_neox': 624, 'n_words': 169, 'path': 'notebooks/Demographic Shares.ipynb', 'repo_name': 'hcorinna/refugee-migration-patterns', 'size': '2269'}