<jupyter_start><jupyter_text>'Real' World ExamplesChristina was nice enough to give me a dataset and some examples in SAS for me to try and duplicate in python. (Most fun I've had in ages...)  Example: Searching for Sickle Cell Anemia in any discharge diagnosis across each discharge record.  This array will pull any record from the statewide outpatient discharge file (opcalendar2017_35) where sickle cell anemia is found across any of the 30 variables for either the Principal Diagnosis (prindiag), 28 Secondary Diagnoses (Diag1-Diag28), or the E-Code (e_code).  If true, the record is exported to the temporary dataset (opsicklecell17); if false, the record is not included in the temporary dataset.  Although there are several individual ICD-10 discharge diagnoses for sickle cell anemia, they all begin with D57.XXX; therefore, the IN: statement captures the sub diagnoses without having to type each one out.Note: the dummy dataset I provided only has 7 diagnosis fields: Prindiag, Diag1-Diag5 and e_code, to limit the size of the file (values in Diag6-Diag28 are rare anyway). ````data opsicklecell17;set op17.opconcalendar2017_35;array dx [30] prindiag diag1-diag28 e_code;	do i = 1 to 30;		if dx(i) in : ('D57') then do;			output opsicklecell17;			leave;	 	end;	end;drop i;run;````<jupyter_code>df = pd.read_excel("PythonExample.xlsx")
df.info()
DIAGS = ["DIAG"+str(diag+1) for diag in range(5)]
DIAGS
df[['PRINDIAG', 'E_CODE'] + DIAGS]
mask = (df['PRINDIAG'].notnull()) & (df['PRINDIAG'].str.contains("D57"))
for col in (['E_CODE'] + DIAGS):
    mask = (mask) | (df[col].notnull()) & (df[col].str.contains("D57"))
mask
df[mask]<jupyter_output><empty_output><jupyter_text>Example: List the Sickle Cell Anemia ED encounter counts by Sex, Race/Ethnicity and Hospital in order of frequency, for Prince George’s County Hospitals````proc freq data= opsicklecell17 order = freq;tables sex race*ethnicit hospid;where hospid in : (210003, 210051, 210060, 210062) and prindiag in : ('D57'); run;````This procedure will provide the encounter counts (in order of most to least) by sex, race/ethnicity and by hospital only occurring in Prince George’s County hospitals (hospid = 210003, 210051, 210060, 210062) AND with a principal discharge diagnosis of Sickle Cell Anemia (ICD-10 code D57).  Cheating...I'm going to cheat and look for some other stuff since there were only two Sickle Cell Anemia cases in this data set.  Chritina suggested I look for * J069 – Acute Respiratory Infection* R0789 – Chest Pain, Other* R51 – Headache* I10 – Essential Hypertension* F1120 – Opioid Dependence<jupyter_code>new_mask = df.PRINDIAG.str.contains('R0789') & df.HOSPID.isin([210003, 210051, 210060, 210062])
df[new_mask].HOSPID.value_counts()
df[new_mask].AGEGRP.value_counts()
df.SEX.value_counts?
df[new_mask].SEX.value_counts()
pd.crosstab(df[new_mask].RACE,df[new_mask].ETHNICIT)
pd.crosstab(df[new_mask].RACE,df[new_mask].ETHNICIT)<jupyter_output><empty_output><jupyter_text>Pivot Table by  MeansChristina asked me to show how to do the mean TOT_CHG by hostpital.This is what I did on the fly:<jupyter_code>for hos in (210003, 210051, 210060, 210062):
    print(df[df.HOSPID == hos].TOT_CHG.mean())<jupyter_output><empty_output><jupyter_text>This is how you should do it in pandas<jupyter_code>df.pivot_table(index=['HOSPID'], values=['TOT_CHG'], aggfunc='mean')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10481251845290818, 'fraction_numerical': 0.051963389430174194, 'mean_word_length': 5.803212851405623, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '52123', 'license': 'MIT', 'n_tokens_mistral': 1287, 'n_tokens_neox': 1125, 'n_words': 399, 'path': '04_Examples.ipynb', 'repo_name': 'kialio/py4Epis', 'size': '3387'}