<jupyter_start><jupyter_text>Interpreting your coefficients with multicollinearity **Multicollinearity** is when one predictor variable in your regression model can be accurately predicted from the others.  Use `import statsmodels.api as sm`, `from statsmodels.tools.tools import add_constant` (provide full summary of the data)+ check for `R-Squared` value.+ check for `Coefficient` of independent variables. These tell us how you can expect the likelihood of being one class to respond to changes in features.+ check for `P-values` which tell us the relative statistical significance of each variables.<jupyter_code>import numpy as np
import pandas as pd
import seaborn as sns<jupyter_output><empty_output><jupyter_text>Data<jupyter_code>iris = sns.load_dataset('iris')
iris.head()
iris['species'].value_counts()
# We will make it into Binary Classification problem (dropping Sentosa)
iris = iris[iris['species'] != 'setosa']

# convert label to 0 and 1
iris['species'] = iris['species'].apply(lambda x: 1 if x=='versicolor' else 0)
iris['species'].value_counts()<jupyter_output><empty_output><jupyter_text>Regression modelling<jupyter_code>X = iris.drop('species', axis=1) 
y = iris['species']<jupyter_output><empty_output><jupyter_text>Use logithttps://pypi.org/project/statsmodels/<jupyter_code>import statsmodels.api as sm
from statsmodels.tools.tools import add_constant
# add constant term
X = add_constant(X)
model = sm.Logit(y, X)
result = model.fit()
print(result.summary())<jupyter_output>Logit Regression Results                           
==============================================================================
Dep. Variable:                species   No. Observations:                  100
Model:                          Logit   Df Residuals:                       95
Method:                           MLE   Df Model:                            4
Date:                Wed, 10 Feb 2021   Pseudo R-squ.:                  0.9142
Time:                        13:20:12   Log-Likelihood:                -5.9493
converged:                       True   LL-Null:                       -69.315
Covariance Type:            nonrobust   LLR p-value:                 1.947e-26
================================================================================
                   coef    std err          z      P>|z|      [0.025      0.975]
--------------------------------------------------------------------------------
const           42.6378     25.708      1.659 [...]<jupyter_text>**Before you interpret those values, we need to understand if multicollinearity is present in our data.**Multicollinearity won't affect the quality of predictions in our model, but only our abiblity to intrepret individual coefficients in p-values. This brings us to `Variance Inflation Factor(VIF)`. ------------   Variance Inflation Factor (VIF) + **Variance Inflation Factor (VIF)** tells us the extent to which we have multicollinearity in our result+ a `factor of 5 to 10 or more is considered high` and tells us to be wary of the model coefficients.<jupyter_code>from statsmodels.stats.outliers_influence import variance_inflation_factor
vif = [variance_inflation_factor(X.values, i) for i in range(X.shape[1])]
vif # this responds for each columns
pd.DataFrame(list(zip(list(X.columns), vif)))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15528137225398736, 'fraction_numerical': 0.021366235329521515, 'mean_word_length': 2.8968347010550994, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 21, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '396931', 'license': 'MIT', 'n_tokens_mistral': 987, 'n_tokens_neox': 888, 'n_words': 297, 'path': 'ML - Mistakes to avoid in Machine Learning/01_12_Coefficients_without_Treating_multicollinearity.ipynb', 'repo_name': 'ptyadana/probability-and-statistics-for-business-and-data-science', 'size': '3323'}