<jupyter_start><jupyter_text>Fund situation for all students** n/a = Could be records with no fund & also records with fund but with no specified type**<jupyter_code>df['fundType'].value_counts().plot(kind='pie')
plt.show()<jupyter_output><empty_output><jupyter_text>Merge TA and RA to RA/TA<jupyter_code>df['fundType'].replace(['RA','TA'],'TA/RA',inplace=True)
df['fundType'].value_counts()<jupyter_output><empty_output><jupyter_text>Overall status<jupyter_code>((df['fund']>0)|(df['fundType']!='n/a')).value_counts().plot(autopct='%1.1f%%',kind='pie')
plt.show()<jupyter_output><empty_output><jupyter_text>** We assume 49% of students didn't get any fund or scholarship.**  Type of funds (for those who gets fund)<jupyter_code>df[df['fund']>0]['fundType'].value_counts()
df[df['fund']>0]['fundType'].value_counts().plot(autopct='%1.1f%%',kind='pie')
plt.show()<jupyter_output><empty_output><jupyter_text>About 63% student get fund from being Prof Assistance.  Grouping FUNDs with Bins<jupyter_code>df[df['fund']==0].count()[1]
df[(df['fund']==0) & (df['fundType']!='n/a')]<jupyter_output><empty_output><jupyter_text>** so 1738 of our records did not get any fund **<jupyter_code>df['fund']=df['fund'].apply(pd.to_numeric, errors='coerce').dropna()
myBins=np.array([df['fund'].min(),1000,2000,3000,4000,5000,10000,25000,50000,100000,df['fund'].max()])
z=pd.cut(df['fund'], myBins)
z.value_counts()
count=df['fund'].value_counts().sort_index()
count=count[count.index!=0]
p=count.plot()
p.set_ylim(0,count.max())
p.set_xlim(0,100000)
plt.show()
z.value_counts().sort_index().plot(kind='bar')
plt.show()
df.rename(columns={'fund': 'fundValue'}, inplace=True)
df.fundValue.count()
df['fund']=df['fundValue']>0
df[['fund','fundValue','fundType']].head()<jupyter_output><empty_output><jupyter_text>Saving data<jupyter_code>df.to_json('Data_FixedFound.json',date_format='utf8')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.21503680336487907, 'fraction_numerical': 0.03680336487907466, 'mean_word_length': 11.771812080536913, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '38229', 'license': 'MIT', 'n_tokens_mistral': 797, 'n_tokens_neox': 742, 'n_words': 96, 'path': '03_DataPreprocessing/06_Fund/Fund.ipynb', 'repo_name': 'yazdipour/DM17', 'size': '1902'}