<jupyter_start><jupyter_text>what does these coeffiecients represent this actually represents the difference in the change in hygiene scores if a person has no musical affiliation,  compared to someone who is a crusty, indie, metal, Rescpectively  Outliers and Influential cases<jupyter_code>summary_ = sms.OLSInfluence(m03).summary_frame()
summary_ = summary_[['cooks_d','standard_resid', 'student_resid', 'hat_diag' ]]
summary_.reset_index(inplace=True, drop=True)
summary_.head()
da_.reset_index(inplace=True, drop=True)

resid = pd.DataFrame(da_['change'] - m03.fittedvalues)
resid.columns = ['residual']
resid.reset_index(inplace=True, drop=True)
dfbeta = pd.DataFrame(pd.DataFrame(sms.OLSInfluence(m03).dfbeta)[0])
dfbeta.columns = ['dfbeta']
cov_ratio = pd.DataFrame(sms.OLSInfluence(m03).cov_ratio)
cov_ratio.columns = ['cov_ratio']
dq = [da_, resid, summary_, dfbeta]
final_summary = reduce(lambda left,right: pd.merge(left,right, left_index=True, right_index=True), dq)
large_resid  = final_summary[(final_summary['standard_resid']>=2) | (final_summary['standard_resid']<=-2)]
large_resid
k = 3 #number of predictors
n = 123 #number of objervations

average_leverage = (k+1)/n
print(average_leverage)
cvr_limit_high = 1+3*average_leverage
cvr_limit_low  = 1-3*average_leverage

print(cvr_limit_low, cvr_limit_high)
print(cov_ratio.iloc[7])
print(cov_ratio.iloc[21])
print(cov_ratio.iloc[26])
print(cov_ratio.iloc[46])
print(cov_ratio.iloc[69])<jupyter_output>cov_ratio    0.892255
Name: 7, dtype: float64
cov_ratio    0.872171
Name: 21, dtype: float64
cov_ratio    0.821091
Name: 26, dtype: float64
cov_ratio    0.848158
Name: 46, dtype: float64
cov_ratio    0.886672
Name: 69, dtype: float64<jupyter_text>Therefore, we are looking for any cases that deviate substantially from these boundaries.. However, given the Cook’s distance for this case, there is probably little cause for alarm.None of them has a Cook’s distance greater than 1, so none of the cases is having an undue influence on the model.  Checking Assumptions  Checking Assumptions of Independent Errors<jupyter_code>print(durbin_watson(m03.resid))<jupyter_output>1.8934071205674472<jupyter_text>Assumption of no multicollinearity<jupyter_code>from statsmodels.tools.tools import add_constant
from statsmodels.stats.outliers_influence import variance_inflation_factor
da_ = add_constant(da_)
dz = da_[['const', 'Crusty', 'Indie', 'Metaller']]
dz.reset_index(drop=True, inplace=True)
vif = pd.Series([variance_inflation_factor(dz.values, i) 
               for i in range(1, dz.shape[1])], 
              index=dz.columns[1:])
print(vif)
avg_vif  = np.mean(vif)
print(avg_vif)
tolerance  = 1/vif
tolerance<jupyter_output><empty_output><jupyter_text>Assumption about the Residuals<jupyter_code>prediction = pd.DataFrame(m03.fittedvalues)
prediction.columns = ['predicted']
prediction.reset_index(drop=True, inplace=True)
print(prediction.head())
prediction['standarized_prediction']  = (prediction['predicted']-prediction['predicted'].mean())/prediction['predicted'].std()
_ = sns.scatterplot(x= final_summary['standard_resid'], y = prediction['standarized_prediction'] )
plt.show()
_ = pg.qqplot(final_summary['standard_resid'],)
plt.show()
fig,ax = plt.subplots(figsize=(6, 4))
ax = plt.hist(final_summary['student_resid'],density=True,bins=30, edgecolor='black', linewidth=1.4)
plt.xlabel('student_resid', fontsize=14)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12430858806404659, 'fraction_numerical': 0.03406113537117904, 'mean_word_length': 7.360097323600973, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 16, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '325110', 'license': 'MIT', 'n_tokens_mistral': 1313, 'n_tokens_neox': 1206, 'n_words': 250, 'path': 'Python/statistics_with_Python/07_Regression/Markdown_notebook/03_categoricalVar_dummyCoding.ipynb', 'repo_name': 'snehilk1312/AppliedStatistics', 'size': '3435'}