<jupyter_start><jupyter_text>Logistic regression  ContextThe original dataset contains 1000 entries with 20 categorial/symbolic attributes prepared by <NAME>. In this dataset, each entry represents a person who takes a credit by a bank. Each person is classified as good or bad credit risks according to the set of attributes. It is almost impossible to understand the original dataset due to its complicated system of categories and symbols. Thus, the attributes in this dataset is a subset of the original dataset. Several columns are simply ignored, and some of the important attritubes like age, account balance etc. are retained.  Dataset: German Credit Risk- Age (Numeric : Age in years)- Sex (Categories : male, female)- Job (Categories : 0 - unskilled and non-resident, 1 - unskilled and resident, 2 - skilled, 3 - highly skilled)- Housing (Categories : own, rent, or free)- Saving accounts (Categories : little, moderate, quite rich, rich)- Checking account (Categories : little, moderate, rich)- Credit amount (Numeric : Amount of credit in DM - Deutsche Mark)- Duration (Numeric : Duration for which the credit is given in months)- Purpose (Categories: car, furniture/equipment, radio/TV, domestic appliances, repairs, education, business, vacation/others)- Risk (0 - Person is not at risk, 1 - Pesron is at risk(defaulter))  ObjectiveThe objective is to estimate probabilities whether a person would default or not using logistic regression. In this dataset, the target variable is 'Risk'.  1. Load Libraries and data<jupyter_code>import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
import matplotlib.pyplot as plt       # matplotlib.pyplot plots data
%matplotlib inline 
import seaborn as sns

# Remove scientific notations and display numbers with 2 decimal points instead
pd.options.display.float_format = '{:,.2f}'.format

pdata = pd.read_csv("German_Credit.csv")
pdata.head()<jupyter_output><empty_output><jupyter_text>2. Calculate the percentage of missing values for each column and fill those missing values.<jupyter_code>totalvals = pd.DataFrame(pdata.count(), columns = ['Number of entries'])
misvals = pd.DataFrame(pdata.isnull().sum(), columns= ['Number of missing values'])
newdf = pd.merge(totalvals, misvals, how = 'outer', right_index = True, left_index=True)
newdf['percmis'] = (newdf['Number of missing values'] / newdf['Number of entries']) * 100
print(newdf)

## This is wrong, teh number of entris for each column is 1000. df.count() counts without the nans
#Correct code:
# df.isna().sum()/len(df.isna())
#the above code will be implemented on all columns even though it is not specified. 
from sklearn.impute import SimpleImputer
rep_0 = SimpleImputer(strategy="most_frequent")
cols=['Saving accounts','Checking account']
imputer = rep_0.fit(pdata[cols])
pdata[cols] = imputer.transform(pdata[cols])

pdata.head()
#pdata.head()


#This is also wrong. Cannot impute unknown categorical data taht is dependant on other factors withy "most frequent"
# Need to imput NaN with unknown as below: 
# df['Saving accounts'].fillna(value='unknown',inplace=True)
# df['Checking account'].fillna(value='unknown',inplace=True)
# df.head()<jupyter_output><empty_output><jupyter_text>3. Plot histogram for columns 'Credit amount' and 'Age'<jupyter_code>plt.figure(figsize=(20,12))
creditplot = sns.barplot(x='Age', y='Credit amount', data=pdata);
creditplot.set_xticklabels(creditplot.get_xticklabels(), rotation=40, ha="right")
plt.tight_layout()

#This is also wrong. Hiostogram to be plotted as below: 
#df['Age'].hist()<jupyter_output><empty_output><jupyter_text>4. Create counplots for columns 'Duration', 'Purpose'. What durations are most common and for what purposes most credits are taken?<jupyter_code>plt.figure(figsize=(20,12))
durplot = sns.countplot(x='Duration', data=pdata)
durplot.set_xticklabels(rotation=40, ha="right")
plt.tight_layout()


#Insight: most common duration - 12 months, 24 months

plt.figure(figsize=(20,12))
purplot = sns.countplot(x='Purpose', data=pdata)
purplot.set_xticklabels(purplot.get_xticklabels(), rotation=40, ha="right")
plt.tight_layout()

#Insight: most common purpose - car or radio/tv<jupyter_output><empty_output><jupyter_text>5. Which 'Job', 'Savings account', 'Checking account' has highest number of defaulters?<jupyter_code>jobdef = pdata.groupby(['Job', 'Risk'])[['Risk']].count()
savdef = pdata.groupby(['Saving accounts','Risk'])[['Risk']].count()
checkdef = pdata.groupby(['Checking account','Risk'])[['Risk']].count()
print(jobdef)
print(savdef)
print(checkdef)

#A better way to do this: 
#pd.crosstab(df['Job'],df['Risk'],normalize=False)
# Interpretation: 
# Job2 has the highest number of defaulters, 
# Those with 'little' money in their savings account are the highest defaulters
# Those with 'little' money in their checking account are the highest defaulters<jupyter_output><empty_output><jupyter_text>6. Convert 'Credit amount' and 'Age' into categorical features by grouping them into different intervals and drop the original columns.You can decide the intervals using df.describe() method.<jupyter_code>pdata.describe()
pdata['Credit amount cut'] = pd.qcut(pdata['Credit amount'], q=4)
pdata['Age cut'] = pd.qcut(pdata['Age'], q=4)
pdata.head()
pdata = pdata.drop(['Credit amount','Age'], axis=1)
# pdata['Job'] = pd.Categorical(pdata.Job)
# pdata['Duration'] =pd.Categorical(pdata.Duration)
# pdata.head()<jupyter_output><empty_output><jupyter_text>7. Find out total number of defaulters where for each category of 'Age' and  'Credit amount'.<jupyter_code>creddef = pdata.groupby(['Credit amount cut', 'Risk'])[['Risk']].count()
agedef = pdata.groupby(['Age cut', 'Risk'])[['Risk']].count()
print(creddef)
print(agedef)<jupyter_output>Risk
Credit amount cut  Risk      
(249.999, 1365.5]  0      173
                   1       77
(1365.5, 2319.5]   0      188
                   1       62
(2319.5, 3972.25]  0      194
                   1       56
(3972.25, 18424.0] 0      145
                   1      105
                     Risk
Age cut        Risk      
(18.999, 27.0] 0      184
               1      107
(27.0, 33.0]   0      151
               1       74
(33.0, 42.0]   0      190
               1       59
(42.0, 75.0]   0      175
               1       60<jupyter_text>8. Prepare input data for the model  Separate dependent and independent variables<jupyter_code>X = pdata.drop('Risk',axis=1)     # Predictor feature columns (9 X m)
Y = pdata['Risk']   # Predicted class (1=True, 0=False) (1 X m)<jupyter_output><empty_output><jupyter_text>One hot enocding for all the categorical variables<jupyter_code>list(pdata.columns)
feature_cols = ['Sex',
 'Job',
 'Housing',
 'Saving accounts',
 'Checking account',
 'Duration',
 'Purpose',
 'Credit amount cut',
 'Age cut']
features = pdata[feature_cols]
features.info()
features_dummies = pd.get_dummies(features)
features_dummies.head()<jupyter_output><empty_output><jupyter_text>Split data into train and test (maintain the ratio of both classes in train and test set same as the original dataset)<jupyter_code>from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test = train_test_split(features_dummies, Y, test_size=0.3, random_state=1)
# 1 is just any random seed number
x_train.head()<jupyter_output><empty_output><jupyter_text>9. Predict Target Variable for Test Data using Sklearn<jupyter_code>from sklearn import metrics

from sklearn.linear_model import LogisticRegression

# Fit the model on train
model = LogisticRegression(solver="liblinear")
model.fit(x_train, y_train)
#predict on test
y_predict = model.predict(x_test)


coef_df = pd.DataFrame(model.coef_)
coef_df['intercept'] = model.intercept_
print(coef_df)

model_score = model.score(x_test, y_test)
print(model_score)<jupyter_output>0         1         2         3         4         5         6  \
0 -0.041797  0.043586 -0.157831 -0.422672 -0.034209 -0.563308  0.017014   

          7         8         9  ...        21        22        23        24  \
0  0.346878  0.174819 -0.051326  ... -0.403625  0.294687 -0.187775 -0.502496   

         25        26        27        28        29  intercept  
0 -0.184918  0.119631  0.249453 -0.603868 -0.345718  -0.580503  

[1 rows x 31 columns]
0.7333333333333333<jupyter_text>10. Try calculating Probabilities of Target using Sklearn<jupyter_code>pred = model.predict_proba(x_train)
print(pred)
preddf = pd.DataFrame(data = y_predict, index = None, columns = ['Risk_pred'])
preddf.head()
p_copy = pdata.copy()
comparedf = pd.merge(p_copy, preddf, how='outer', left_index=True, right_index=True)
comparedf.head(20)<jupyter_output><empty_output><jupyter_text>11. Build a confusion matrix and display the same using heatmap plot<jupyter_code>cm=metrics.confusion_matrix(y_test, y_predict, labels=[1, 0])

df_cm = pd.DataFrame(cm, index = [i for i in ["1","0"]],
                  columns = [i for i in ["Predict 1","Predict 0"]])
plt.figure(figsize = (10,8))
sns.heatmap(df_cm, annot=True)<jupyter_output><empty_output><jupyter_text>12.  Make a predicted probability distribution plot for defaulters and non defaulters<jupyter_code>preddf.head()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11533875338753388, 'fraction_numerical': 0.043685636856368566, 'mean_word_length': 4.395321637426901, 'pattern_counts': {'":': 0, '<': 56, '<?xml version=': 0, '>': 56, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '665640', 'license': 'MIT', 'n_tokens_mistral': 3274, 'n_tokens_neox': 2959, 'n_words': 903, 'path': 'Credit Risk Project/Credit Risk_Logistic Regression.ipynb', 'repo_name': 'RevatiDineshDeshpande/machine-learning-projects', 'size': '9232'}