<jupyter_start><jupyter_text>In this lab exercise, we will become familiar with regression analysis. We will learn how to do descriptive regression analysis in python using the statsmodels library. Reference:- More information on formula syntax: https://patsy.readthedocs.io/en/latest/formulas.html- More information about statsmodels: https://www.statsmodels.org/dev/example_formulas.html<jupyter_code>import pandas as pd
import numpy as np
import statsmodels.formula.api as smf

# set seed for consistency
np.random.seed(2)<jupyter_output><empty_output><jupyter_text>Data - previewWe will use a dataset with information about patients with cardiovascular diseases. Here is the breakdown of columns:Some variables are categorical:- 'DEATH_EVENT': if the patient deceased during the follow-up period- 'sex': male/female (binary, male 1, female 0)- 'anaemia': decrease of red blood cells (boolean, yes 1)- 'smoking': if the patient smokes or not (boolean, yes 1)- 'diabetes': if the patient has diabetes (boolean, yes 1)- 'high_blood_pressure': if the person has hypertension (boolean, yes 1)... and some are continuous/discrete:- 'age': patient's age- 'creatinine_phosphokinase': level of CPK enzyme in the blood (mcg/L)- 'ejection_fraction': percentage of blood leaving the heart in each contraction- 'platelets': platelets in the blood (kiloplatelets/mL)- 'serum_creatinine': level of serum creatinine in the blood (mg/dL)- 'serum_sodium': level of serum sodium in the blood (mEq/L)- 'time': duration of the follow-up period (number of days in the hospital)<jupyter_code>df = pd.read_csv('data/heart_failure_clinical_records_dataset.csv')
df<jupyter_output><empty_output><jupyter_text>Linear regression: Modeling days spent in hospital- Using linear regression, we will model the number of days spent in the hospital from the patient population.- We need two components to do that:   1. An equation describing a model   2. Data   - Equations are specified using [syntax] (https://patsy.readthedocs.io/en/latest/formulas.html) patsy software package:    1. `~`: Separates the left and right sides    2. `+`: Creates a union of predictors used    3. `:`: Interaction term    3. `*`: `a * b` is a shorter notation for` a + b + a: b`, and is often useful when we want to include all interactions between a set of variables        - Interceptions are added automatically- Category variables are added using the term C(a).- We can use pandas for data! Example- Let's start with an example from our dataset. We are interested in two predictors: diabetes and high blood pressure. These are the two predictors we want to use to fit the outcome, the number of days spent in the hospital, using linear regression.- The model that achieves this is formulated as:        time ~ C(diabetes) + C(high_blood_pressure)        - We can create this model using smf.ols()- OLS stands for simple linear least squares regression- Two components: the formula and the data are explicitly stated.- The terms in the formula are columns in the pandas data frame. Easy!<jupyter_code># Declares the model
mod = smf.ols(formula='time ~ C(diabetes) + C(high_blood_pressure)', data=df)
# Fits the model (find the optimal coefficients, adding a random seed ensures consistency)
res = mod.fit()
# Print thes summary output provided by the library.
res.summary()<jupyter_output><empty_output><jupyter_text>A lot of useful information is provided by default.- The dependent variable : time (number of days at the hospital)- Method: The type of model that was fitted (OLS)- Nb observations: The number of datapoints (299 patients)- R2: The fraction of explained variance- A list of predictors- For each predictor: coefficient, standard error of the coefficients, p-value, 95% confidence intervals. We can see that only high blood pressure is a significant predictor (p = 0.001), while diabetes is not (0.584).- Warnings if there are numerical issues (hopefully not!)  We can now interpret the learned model$time = 139.3851 + 4.9059 * diabetes + (-31.8228) * highBloodPressure$- What is the expected time if the patient does not have diabetes or high blood pressure?- What if he doesn't have diabetes and he has high blood pressure?- What if he has both?1. People who do not have diabetes or high blood pressure stay in the hospital for an average of 139 days2. People who have diabetes but do not have blood pressure stay 139 + 4.9 days ~ 144 days3. People who do not have diabetes but have blood pressure stay 139 - 31 days ~ 108 days4. People with diabetes and blood pressure stay 139 + 4.9 - 31.8 ~ 112 days- How else could we calculate that?<jupyter_code>selected = df.loc[(df['diabetes'] == 0) & (df["high_blood_pressure"] == 0)]

selected['time'].mean()<jupyter_output><empty_output><jupyter_text>- Nije li čudno da visoki tlak ima negativan koeficijent? Čini se da pacijenti s visokim tlakom ostaju u bolnici kraći broj dana, iako bi se očekivalo suprotno. Znali li netko možda zašto bi to bilo tako?  Linearna regresija: Modeliranje dana provedenih u bolnici V2- Jedan od razloga zašto se ozbiljna stanja mogu povezati s kraćim vremenom provedenim u bolnici je treći čimbenik: smrt 💀. Pacijenti koji imaju ozbiljno stanje mogli bi provesti manje vremena u bolnici jer umiru.- Dobijmo bolji osjećaj što se događa modelirajući vrijeme provedeno u bolnici sa smrću kao prediktorom.- Ovaj put ćemo dodati interakcijske značajke.<jupyter_code># we use a*b to add terms: a, b, a:b, and intercept

mod = smf.ols(formula='time ~ C(high_blood_pressure) * C(DEATH_EVENT,  Treatment(reference=0)) + C(diabetes)',
              data=df)

# C(DEATH_EVENT,  Treatment(reference=0)) implies that we are considering the population that did not die!

res = mod.fit()
res.summary()<jupyter_output><empty_output><jupyter_text>Interpretation- This model lets us see that death is associated with a lower number of days spent at the hospital.- Notice how R2 is much larger compared to the previous model: more variance in the data is explained.- Those that have high blood pressure stay for less time (-26 days on average), those who have blood pressure __and__ die spend 2.7 days more hospitalized on average, although this is not statistically significant.  Standardization<jupyter_code>formula = 'time ~ age + C(high_blood_pressure)'
mod = smf.ols(formula=formula, data=df)
res = mod.fit()
res.summary()<jupyter_output><empty_output><jupyter_text>What can we conclude from the intercept?> that patients who do not have high blood pressure and are zero years remain in the hospital for ~ 223 days<jupyter_code># how we standardize the countinuous variables
columns_to_standardize = [
    "age",
    "creatinine_phosphokinase",
    "ejection_fraction",
    "platelets",
    "serum_creatinine",
    "serum_sodium"
]

for col in columns_to_standardize:
    df[col] = (df[col] - df[col].mean()) / df[col].std()  # standardize column

formula = 'time ~ age + C(high_blood_pressure)'
mod = smf.ols(formula=formula, data=df)
res = mod.fit()
res.summary()<jupyter_output><empty_output><jupyter_text>What can we conclude from the intercept now?> that patients who do not have high blood pressure and have an average number of years remain in the hospital for ~ 140 days  Logaritamska transformacija<jupyter_code>df = pd.read_csv('data/heart_failure_clinical_records_dataset.csv')

df["logtime"] = np.log(df["time"])
mod = smf.ols(formula='logtime ~ C(high_blood_pressure) * C(DEATH_EVENT,  Treatment(reference=0)) + C(diabetes)',
              data=df)
res = mod.fit()
res.summary()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08310175995765516, 'fraction_numerical': 0.012703453751488685, 'mean_word_length': 4.84984520123839, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 35, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '407147', 'license': 'Apache-2.0', 'n_tokens_mistral': 2345, 'n_tokens_neox': 2167, 'n_words': 951, 'path': 'labosi/lab-05/2021-22/by_sheriffHorsey/lab5_engl.ipynb', 'repo_name': 'studosi-fer/UUZOP', 'size': '7557'}