<jupyter_start><jupyter_text>Linear Regression, ANOVA & Collinearity for the Diabetes Dataset  In this video we will be relating HbA1c followup to all other features and discussing collinearity<jupyter_code>import seaborn as sns
import pandas as pd
%matplotlib inline
import matplotlib.pyplot as plt
import scipy.stats as stats
import warnings
warnings.simplefilter("ignore")
# read in the data from a website in CSV format using Pandas

df = pd.read_csv("diabetes_treatment.csv")
df.head()
print(df.info()) # print out the columns and their types and number in reries format
df.describe()<jupyter_output><empty_output><jupyter_text>Creating categories and formatting the dataset<jupyter_code># chaning the name for these features to avoid a naming convention later

df['HbA1c_Followup'] = df['HbA1c.Followup']
df['HbA1c_Baseline'] = df['HbA1c.Baseline']
df.head()
# creating a binary column for Gender

df["Gender"] = df["Female"].astype(int)
df.head()<jupyter_output><empty_output><jupyter_text>Creating categorical variables from continuous data  https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.qcut.html https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.cut.html<jupyter_code># example of using qcut to segment or categorize a data column

pd.Series(pd.qcut(range(12), 3)).value_counts()
# separating into 3 categories

pd.qcut(df['BMI'], q=3).value_counts()
# creating BMI categories

df['BMI_Category'] = pd.qcut(df['BMI'], q=3, precision = 0)
df.head()
# using cut to create BMI categories

df['BMI_Category'] = pd.cut(df['BMI'], bins=3)
df.head()
# selecting the cut ranges for consistency with the R video

cat_bins =[0.0, 30.0, 35.0, 40.0]
df['BMI_Category'] = pd.cut(df['BMI'], bins=cat_bins)
df.head()<jupyter_output><empty_output><jupyter_text>ANOVA test on categorial BMI and other features  There are many ways to do ANOVA testing in Python - here is one way similar to R<jupyter_code>import statsmodels.api as sm
from statsmodels.formula.api import ols
from scipy import stats
# In this summary of followup against all other features we see that the p-values for the BMI_categories
# have values greater than 0.05 which implies no significance.
# They therefore do not significantly predict the HbA1c.followup level or output when controlling for 
# or adjusting for other input features. 
# In this case prediction means that these categories do not explain the variance in the HbA1c.followup level.
#NOte - BMI categories do not include the reference BMI category 0-30 range. 

stats=ols('HbA1c_Followup ~ HbA1c_Baseline + Age + Gender + BMI_Category + SBP', data=df).fit()
print(stats.summary())
# For the ANOVA tests the BMI categories are summarized with the total BMI
# Once again we see that the p-value for the BMI_category is much greater than 0.05 and hence not significant.

stats=ols('HbA1c_Followup ~ HbA1c_Baseline + Age + Gender + BMI_Category + SBP', data=df).fit() 
print(sm.stats.anova_lm(stats, typ=2))<jupyter_output><empty_output><jupyter_text>Collinearity  If you put 2 corrleated variables into a model they will compete to explain variation in the model output<jupyter_code># Previously we saw that when we combined all the other features with BMI, including HbA1c_Baseling, 
# the BMI did not significantly predict the output or explain the variance in the output. 
# However, let's see what happens when we just use BMI by itself as a predictor of the HbA1c.Followup levels.
# Here we see that the regression coeficient is sizable and the p-value is very significant implying 
# that BMI is of significant importance in predicting HbA1c.Followup

stats=ols('HbA1c_Followup ~ BMI', data=df).fit() 
print(stats.summary())
# But what if BMI is controlling or adjusting for HbA1c_Baseline?

stats=ols('HbA1c_Followup ~ HbA1c_Baseline + BMI', data=df).fit() 
print(stats.summary())<jupyter_output><empty_output><jupyter_text>Conclusion - while BMI related by iteself to HbA1c_followup is significant in predicting output (p-value~0) if controlled from where the patient's started i.e. Baseline, it is not significantly related to HbA1c_followup Effectively the two input features HbA1c_Baseline and BMI are collinear features.<jupyter_code>stats=ols('HbA1c_Followup ~ HbA1c_Baseline + BMI', data=df).fit() 
print(sm.stats.anova_lm(stats, typ=2))<jupyter_output><empty_output><jupyter_text>Linear Regression using Sci-kit Learn<jupyter_code>import pandas as pd

df = pd.read_csv("diabetes_treatment.csv")
df.head()
df["Gender"] = df["Female"].astype(int)
df.head()
[col for col in df.columns]
features = ['HbA1c.Baseline']
#features = ['HbA1c.Baseline', 'BMI', 'Age', 'SBP', 'Gender']
target = ['HbA1c.Followup']
x = df[features]
y = df[target]
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error, r2_score
x_train, x_test, y_train, y_test = train_test_split(x,y)
linrgr = LinearRegression()
linrgr.fit(x_train, y_train)
linrgr.score(x_test, y_test)
# model prediction for the test data input
y_pred = linrgr.predict(x_test)

# Here are the regression coefficients
print('Regression Coeffs:\n', linrgr.coef_)

# The coefficient of determination is an indicator of model performance or how much features explain target variance
print('Coeff of Determination - R^2:\n', r2_score(y_test, y_pred))

# The mean squared error indicates the error between the model predictions and the ground-truth data
print('MSE: \n', mean_squared_error(y_test, y_pred))
import seaborn as sns
import matplotlib.pyplot as plt
%matplotlib inline

#sns.scatterplot(df['BMI'], df_diabetes['HbA1c.Followup']);

sns.scatterplot(df['HbA1c.Baseline'], df['HbA1c.Followup']);
plt.plot(x_test, y_pred, color='green', linewidth=2)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09698015530629853, 'fraction_numerical': 0.011216566005176877, 'mean_word_length': 5.983132530120482, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '424201', 'license': 'MIT', 'n_tokens_mistral': 2000, 'n_tokens_neox': 1825, 'n_words': 617, 'path': 'Linear Regression in HealthCare/DC-Multicollinearity-Python-Jamieson.ipynb', 'repo_name': 'Aboubacar2012/Darthmouth_DataScience_HealthCare', 'size': '5795'}