<jupyter_start><jupyter_text>Warmup Question 3, Time Series Analysis:- Based on the attribute **ClaimStartDt**, analyze the weekly service volume variations.<jupyter_code>from datetime import datetime
data['ClaimStartDt'] = pd.to_datetime(data['ClaimStartDt'] , format = '%Y-%m-%d')
dir(data['ClaimStartDt'].dt)
data['ClaimDayName'] = data['ClaimStartDt'].dt.day_name()
data['ClaimStartWeek'] = data['ClaimStartDt'].dt.week
weeklyVol = data.groupby(['ClaimStartWeek','Status'])['ClaimID'].count().reset_index()
data.State
weeklyVol
weeklyVol['logVol'] = np.log(weeklyVol.ClaimID)
plt.figure(figsize=(12,8))


ax1 = plt.subplot(211)
sns.lineplot(x = 'ClaimStartWeek', y = 'ClaimID', hue = 'Status', data = weeklyVol)



ax1 = plt.subplot(212)
sns.lineplot(x = 'ClaimStartWeek', y = 'ClaimID', hue = 'Status', data = weeklyVol[weeklyVol.ClaimStartWeek > 45])<jupyter_output><empty_output><jupyter_text>- What are the weekly patterns (weekly seasonalities) you observe for the demands on **inpatient** or **outpatient**services? Sharp drop around Christmas - Identify the top $5$ most frequent **ClmAdmitDiagnosisCodes** and trace their weekly seasonalities.Display the patterns you find.<jupyter_code>diagCodes = data[['ClmDiagnosisCode_1', 'ClmDiagnosisCode_2', 'ClmDiagnosisCode_3',
       'ClmDiagnosisCode_4', 'ClmDiagnosisCode_5', 'ClmDiagnosisCode_6',
       'ClmDiagnosisCode_7', 'ClmDiagnosisCode_8', 'ClmDiagnosisCode_9',
       'ClmDiagnosisCode_10', 'ClaimID']]
diagMelt = diagCodes.melt(id_vars = 'ClaimID').dropna(axis=0)
diagCounts = diagCodes.melt(id_vars = 'ClaimID').dropna(axis=0)['value'].value_counts().sort_values(ascending=False)
top5 = diagCounts.head(5).index.values
diagCounts
diagTop5 = data[['ClmDiagnosisCode_1', 'ClmDiagnosisCode_2', 'ClmDiagnosisCode_3',
       'ClmDiagnosisCode_4', 'ClmDiagnosisCode_5', 'ClmDiagnosisCode_6',
       'ClmDiagnosisCode_7', 'ClmDiagnosisCode_8', 'ClmDiagnosisCode_9',
       'ClmDiagnosisCode_10', 'ClaimID','ClaimStartWeek']]
diagMelt = diagTop5.melt(id_vars = ['ClaimID','ClaimStartWeek'], var_name = '#', value_name = 'DiagCode').dropna(axis=0)
diag5 = diagMelt[diagMelt.DiagCode.isin(top5)].groupby(['DiagCode','ClaimStartWeek'])['ClaimID'].count().reset_index()
diag5['logVol'] = np.log(diag5.ClaimID)
plt.figure(figsize=(10,10))


ax1 = plt.subplot(211)
ax1 = sns.lineplot(x = 'ClaimStartWeek', y = 'ClaimID', hue = 'DiagCode', data = diag5)
plt.tight_layout()
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)


ax2 = plt.subplot(212)
ax2 = sns.scatterplot(x = 'ClaimStartWeek', y = 'ClaimID', 
                   hue = 'DiagCode', 
                   data = diag5[diag5.ClaimStartWeek.isin(range(15,22,1))])
plt.tight_layout()
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)<jupyter_output><empty_output><jupyter_text>- For the inpatient patients, is there any pattern between the durations of the treatments andthe patient attributes?Race, Age, Gender, NumChronics<jupyter_code>inpat = data[data.Status=='in']
inpat.head()
meanDays = inpat.groupby(['Age','Gender'])['AdmissionDays','ClaimDays'].mean().reset_index()

sns.lineplot(x = 'Age',y = 'AdmissionDays', hue = 'Gender', data = meanDays)
meanDays


meanDays = inpat.groupby(['Age','Gender'])['AdmissionDays','ClaimDays'].mean().reset_index()


d = meanDays.melt(id_vars = ['Age','Gender'])
plt.figure(figsize=(10,10))


ax1 = plt.subplot(311)
ax1 = sns.lineplot(x = 'Age',y = 'value', hue = 'variable',data = d[d.Gender == 0])
plt.tight_layout()
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)

ax1 = plt.subplot(312)
ax1 = sns.lineplot(x = 'Age',y = 'value', hue = 'variable',data = d[d.Gender == 1])
plt.tight_layout()
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)<jupyter_output><empty_output><jupyter_text>- For the monthly inpatient/outpatient claim counts of all the providers, please analyze thetable of provider-month matrices and report any interesting findings.- **hint**: dataframe's pivot_table method could be used to construct the provider-month matrix.<jupyter_code>census = pd.read_csv('./data/census/2017county_data.csv')
census.head()
state_dict = census.groupby('State')['County'].nunique().reset_index()
state_dict.index = range(1,len(state_dict)+1)

state_dict
d= {1: 'Alabama',
 2: 'Alaska',
 3: 'Arizona',
 4: 'Arkansas',
 5: 'California',
 6: 'Colorado',
 7: 'Connecticut',
 8: 'Delaware',
 9: 'District of Columbia',
 10: 'Florida',
 11: 'Georgia',
 12: 'Hawaii',
 13: 'Idaho',
 14: 'Illinois',
 15: 'Indiana',
 16: 'Iowa',
 17: 'Kansas',
 18: 'Kentucky',
 19: 'Louisiana',
 20: 'Maine',
 21: 'Maryland',
 22: 'Massachusetts',
 23: 'Michigan',
 24: 'Minnesota',
 25: 'Mississippi',
 26: 'Missouri',
 27: 'Montana',
 28: 'Nebraska',
 29: 'Nevada',
 30: 'New Hampshire',
 31: 'New Jersey',
 32: 'New Mexico',
 33: 'New York',
 34: 'North Carolina',
 35: 'North Dakota',
 36: 'Ohio',
 37: 'Oklahoma',
 38: 'Oregon',
 39: 'Pennsylvania',
 41: 'Rhode Island',
 42: 'South Carolina',
 43: 'South Dakota',
 44: 'Tennessee',
 45: 'Texas',
 46: 'Utah',
 47: 'Vermont',
 49: 'Virginia',
 50: 'Washington',
 51: 'West Virginia',
 52: 'Wisconsin',
 53: 'Wyoming',
 54: 'Puerto Rico'}
data['State'] = data['State'].apply(lambda x: d[x])
data[data.State=='Hawaii']['County'].sort_values().unique()
data.groupby('State')['County'].nunique().reset_index()
data[['State','County']]
plt.figure(figsize=(6,8))

ax1 = plt.subplot(211)
ax1 = sns.scatterplot(x='Age',y='ClaimID',
                      hue=hue_feature,
                      data=age_claims[age_claims.Age >= 65])

ax1.set(ylabel='Number of Claims')
plt.tight_layout()
ax2 = plt.subplot(212)
ax2 = sns.scatterplot(x='Age',y=np.log(age_claims.ClaimID),
                      hue=hue_feature,
                      data=age_claims[age_claims.Age >= 65])

ax2.set(ylabel='Number of Claims')
plt.tight_layout()<jupyter_output><empty_output><jupyter_text>Study the relationship between the patient age and their chornic conditions. - Within the train-samples, do these chronic conditions show a definite trend with respect to increasing ages?<jupyter_code>age_chronics = data.groupby(['Age','Set'])['Alzheimer', 'HeartFailure', 'KidneyDisease',
       'Cancer', 'ObstrPulmonary', 'Depression', 'Diabetes', 'IschemicHeart',
       'Osteoporasis', 'RheumatoidArthritis', 'Stroke'].sum().reset_index()
age_chronics = age_chronics.melt(id_vars=['Age','Set'])


plt.figure(figsize=(8,8))

ax1 = plt.subplot(211)
ax1 = sns.lineplot(x='Age',y='value',
                      hue='variable',
                      data=age_chronics[age_chronics.Set=='Train'])

ax1.set(ylabel='Cases of Chronic Disease')
plt.tight_layout()
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)

ax1.figure.savefig("./visualizations/aiko/age_Chronic.png",bbox_inches='tight')
ax2 = plt.subplot(212)
ax2 = sns.lineplot(x='Age',y=np.log(age_chronics.value),
                      hue='variable',
                      data=age_chronics[age_chronics.Set=='Train'])

ax2.set(ylabel='Cases of Chronic Disease')
plt.tight_layout()
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)

ax2.figure.savefig("./visualizations/aiko/age_logChronic.png",bbox_inches='tight')<jupyter_output><empty_output><jupyter_text>Yes they show the same trend as the number of claims  In order to make sure the insurance premiums can cover the claims, the insurance company would need to categorize the patients according to their resource usage. In answering the question of what types of patients would make more outpatient visits, please provide your findings. Not sure how best to tackle this. Going with the theme of Age and adding Gender and Number of Chronic Diseases...<jupyter_code>#out = data[data.Status=='out']
arg = data.groupby(['Age','Gender','Race'])['BeneID'].nunique().reset_index()
arg
arg_ = arg.melt(id_vars=['Age','Gender','Race'], value_vars=['BeneID'])
arg_
plt.figure(figsize=(8,8))

ax1 = plt.subplot(211)
ax1 = sns.lineplot(x = 'Age',
                   y='BeneID',
                   hue='Race',
                   style='Gender'
                   estimator=None, lw=1,
                   data=arg)

ax1.set(ylabel='Number of Beneficiaries')
plt.tight_layout()
#plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)

#ax1.figure.savefig("./visualizations/aiko/age_Bene.png",bbox_inches='tight')
ax2 = plt.subplot(212)
ax2 = sns.lineplot(x = 'Age',
                   y='BeneID',
                   hue='Race',
                   style='Gender'
                   estimator=None, lw=1,
                   data=arg[arg.Age>60])

ax2.set(ylabel='Number of Beneficiaries')
plt.tight_layout()
#plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)

#ax2.figure.savefig("./visualizations/aiko/age_Bene60.png",bbox_inches='tight')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16723433242506813, 'fraction_numerical': 0.03303814713896458, 'mean_word_length': 5.01296928327645, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '531785', 'license': 'MIT', 'n_tokens_mistral': 3504, 'n_tokens_neox': 3121, 'n_words': 620, 'path': 'sam/notebooks/Aiko-Q3.ipynb', 'repo_name': 'DebLeong/HealthCareCapstone', 'size': '8808'}