<jupyter_start><jupyter_text>*German credit data* - analysis  Libraries import<jupyter_code>import pandas as pd
import numpy as np
import sklearn 
import seaborn as sns
import matplotlib.pyplot as plt
import warnings
warnings.filterwarnings('ignore')<jupyter_output><empty_output><jupyter_text>Read data  read data online<jupyter_code>data = pd.read_csv("https://www.mldata.io/download-csv-weka/german_credit_data/")
atributes = pd.read_csv("https://www.mldata.io/download-attributes/german_credit_data/")<jupyter_output><empty_output><jupyter_text>Intro data analysis<jupyter_code>data.telephone = data.telephone.map({"A191": 0, "A192": 1})
data.foreign_worker = data.foreign_worker.map({"A201": 1, "A202": 0})<jupyter_output><empty_output><jupyter_text>Several variables underwent type change due to their binariness  ConclusionWithout missing data<jupyter_code>data.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 1000 entries, 0 to 999
Data columns (total 21 columns):
checking_account_status    1000 non-null object
duration                   1000 non-null int64
credit_history             1000 non-null object
purpose                    1000 non-null object
credit_amount              1000 non-null float64
savings                    1000 non-null object
present_employment         1000 non-null object
installment_rate           1000 non-null float64
personal                   1000 non-null object
other_debtors              1000 non-null object
present_residence          1000 non-null float64
property                   1000 non-null object
age                        1000 non-null float64
other_installment_plans    1000 non-null object
housing                    1000 non-null object
existing_credits           1000 non-null float64
job                        1000 non-null object
dependents                 1000 non-null int64
telephone                  [...]<jupyter_text>Conclusions1. There is only 3.7% workers, who work in foreign country2. Dataset is not balanced (70% 'yes' answers)3. Credit_amount has significant outliers (check boxplots below)<jupyter_code>num_var = ['duration', 'credit_amount', 'age', 'existing_credits', 'dependents', 'telephone', 'foreign_worker']
data[num_var].describe()
data.customer_type.replace([1,2], [1,0], inplace=True)
data.customer_type.value_counts()<jupyter_output><empty_output><jupyter_text>Map variables<jupyter_code>data.job = data.job.map({"A171" : "unemployed/unskilled","A172": "unskilled-resident", "A173": "skilled_employee","A174": "management/self-employed"})
data.housing = data.housing.map({'A151' : 'rent', 'A152' : 'own', 'A153' : 'free'})
data.purpose = data.purpose.map({'A40': 'new car', 'A41': 'used car', 'A42': 'furniture/equipment', 'A43': 'radio/television', 'A44': 'domestic appliances', 'A45': 'repairs', 'A46': 'education', 'A47': 'vacation', 'A48':'retraining',
'A49': 'business', 'A410': 'others'})
data.other_debtors = data.other_debtors.map({'A101': 'none', 'A102': 'co-applicant', 'A103': 'guarantor'})
data.other_installment_plans = data.other_installment_plans.map({'A141': 'bank', 'A142': 'stores', 'A143': 'none'})
data.head(10)<jupyter_output><empty_output><jupyter_text>Correlation matrix<jupyter_code>corr=data[num_var].iloc[:,:].corr()

ax=sns.heatmap(corr,
               xticklabels=corr.columns,
               yticklabels=corr.columns, annot=True)

bottom, top = ax.get_ylim()
ax.set_ylim(bottom + 0.5, top - 0.5)
ax.set_title('Variables correlations')
plt.show()<jupyter_output><empty_output><jupyter_text>Conclusions1. Credit amount and duration are strongly correlated (as expected)  Advance analysis  Age distribution Conclusions 1. Credit is mostly taken by 30-year-olds *(plot 1)*2. Reliabilty doesn't depend on the client's age  *(plot 2)*<jupyter_code>sns.distplot(data.age)
plt.title("Age distribution")
ax1 = data[data.customer_type==1].age.plot(kind='hist')
ax2 = data[data.customer_type==0].age.plot(kind='hist')
plt.title('Age distribution by customer type')<jupyter_output><empty_output><jupyter_text>Duration distribution Conclusions 1. The distribution is irregular *(plot 1)*2. Credits were most frequently taken for a period of 12 months *(plot 1)*3. Credits taken for a shorter time were paid back eariler *(plot 2)*<jupyter_code>sns.distplot(data.duration, bins=8)
plt.title("Duration distribution")
ax1 = data[data.customer_type==1].duration.plot(kind='hist', bins=10)
ax2 = data[data.customer_type==0].duration.plot(kind='hist', bins=10)
plt.title('Duration distribution by customer type')<jupyter_output><empty_output><jupyter_text>Credit amounts Conclusions 1. Credit amount and duration are strongly correlated (as expected) *(plot 1)*2. The higher taken credits were, the earlier they were paid back *(plot 3)*<jupyter_code>sns.lmplot(y = 'credit_amount', x = 'duration', hue='customer_type', data = data)
plt.title("Credit amount - duration")
sns.distplot(data.credit_amount)
plt.title("Credit amount distribution")
ax1 = data[data.customer_type==1].credit_amount.plot(kind='hist')
ax2 = data[data.customer_type==0].credit_amount.plot(kind='hist')
plt.title('Duration distribution by customer type')<jupyter_output><empty_output><jupyter_text>Needed functions<jupyter_code>def plot_countplot(col_name: str, y_lim: int):
    fig, ax =plt.subplots(1,2)
    sns.countplot(x=col_name, data=data[data.customer_type==0], order=data[col_name].value_counts().index, ax=ax[0])
    sns.countplot(x=col_name, data=data[data.customer_type==1], order=data[col_name].value_counts().index, ax=ax[1])
    fig.show()
    plt.setp(ax, ylim=[0, y_lim])
    plt.title(col_name + " (No/Yes)")
    plt.show()
    
def cal_mean_val(col_name: str):
    return data.groupby(col_name)["customer_type"].mean()<jupyter_output><empty_output><jupyter_text>Personal info Analysis Conclusions 1. There has been no single female clients2. *A91* doesn't strictly follow 70/30 distribution rule  Personal Marital Status and Sex 1. *A91* : male : divorced/separated2. *A92* : female : divorced/separated/married 3. *A93* : male : single 4. *A94* : male : married/widowed5. *A95* : female : single<jupyter_code>male_divsep = data[data.personal=='A91']
female_divsepmarr = data[data.personal=='A92']
male_single = data[data.personal=='A93']
male_marrwid = data[data.personal=='A94']
female_single = data[data.personal=='A95']
female_single.customer_type.value_counts()
plot_countplot('personal', 420)
cal_mean_val('personal')<jupyter_output><empty_output><jupyter_text>Credit history Conclusions 1. Contrary to expectations, clients, who have paid previous credits duly, don't do so now - and vice-versa 1. *A30*: no credits taken/ all credits paid back duly 2. *A31*: all credits at this bank paid back duly 3. *A32*: existing credits paid back duly till now 4. *A33*: delay in paying off in the past 5. *A34* : critical account/ other credits existing (not at this bank)<jupyter_code>plot_countplot('credit_history', 370)
cal_mean_val('credit_history')<jupyter_output><empty_output><jupyter_text>Checking account status Conclusions1. Clients who didn't possess a checking account, or had significant salary assigned to it were most likely to pay the credit.2. Probability of paying credit or not was most likely even for clients with < 200 DM 1. *A14*: no checking account  2. *A11*: < 0 DM  3. *A12*: 0 <= x < 200 DM  4. *A13*: >= 200 DM   **salary assignments for at least 1 year**<jupyter_code>plot_countplot('checking_account_status', 380)
cal_mean_val('checking_account_status')<jupyter_output><empty_output><jupyter_text>Purpose Conclusions1. What's interesting used car credit was much more likely to be paid than new car credit.2. 'Others' least likely to get paid(hard to tell the reason)3. Business, education less likely to get paid(maybe struggles of beginning a new life?)<jupyter_code>col_name='purpose'
fig, ax =plt.subplots(2,1)
sns.countplot(y=col_name, data=data[data.customer_type==0], order=data[col_name].value_counts().index, ax=ax[0])
sns.countplot(y=col_name, data=data[data.customer_type==1], order=data[col_name].value_counts().index, ax=ax[1])
fig.show()
plt.title(col_name + " (No/Yes)")
plt.show()
cal_mean_val('purpose')<jupyter_output><empty_output><jupyter_text>Savings Conclusions1. The wealthier the client the probability of paying the credit is higher.2. Also clients with unknown savings were most likely to pay it. 1. *A61*: < 100 DM  2. *A62*: 100 <= x < 500 DM  3. *A63*: 500 <= x < 1000 DM  4. *A64*: >= 1000 DM  5. *A65*: unknown/ no savings account<jupyter_code>plot_countplot('savings', 400)
cal_mean_val('savings')<jupyter_output><empty_output><jupyter_text>Present employment Conclusions1. What is installment rate exactly? 1. *A71*: unemployed  2. *A72*: < 1 year  3. *A73*: 1 <= x < 4 years  4. *A74*: 4 <= x < 7 years  5. *A75*: .. >= 7 years<jupyter_code>plot_countplot('present_employment', 250)
cal_mean_val('present_employment')<jupyter_output><empty_output><jupyter_text>Installment rate<jupyter_code>plot_countplot('installment_rate', 330)
cal_mean_val('installment_rate')<jupyter_output><empty_output><jupyter_text>Other debtors Conclusions1. Better clients had a guarantor, worse had a co-applicant<jupyter_code>plot_countplot('other_debtors', 700)
cal_mean_val('other_debtors')<jupyter_output><empty_output><jupyter_text>Property ConclusionsPeople pay duly if they have a property 1. *A121*: real estate2. *A122*: if not A121: building society savings agreement/ life insurance3. *A123*: if not A121/A122 : car or other, not in attribute 64. *A124*: unknown / no property<jupyter_code>plot_countplot('property', 250)
cal_mean_val('property')<jupyter_output><empty_output><jupyter_text>Foreign worker<jupyter_code>plot_countplot('foreign_worker', 680)
cal_mean_val('foreign_worker')<jupyter_output><empty_output><jupyter_text>Other installment plans<jupyter_code>plot_countplot('other_installment_plans', 600)
cal_mean_val('other_installment_plans')<jupyter_output><empty_output><jupyter_text>Housing Conclusion1. People, who have own house is more cases pay credit<jupyter_code>plot_countplot('housing', 550)
cal_mean_val('housing')<jupyter_output><empty_output><jupyter_text>Jobs Conclusions1. Nothing special<jupyter_code>plot_countplot('job', 450)
cal_mean_val('job')<jupyter_output><empty_output><jupyter_text>Exisitng credits number of existing credits at this bank Conclusions1. Nothing special<jupyter_code>plot_countplot('existing_credits', 450)
cal_mean_val('existing_credits')<jupyter_output><empty_output><jupyter_text>Dependents Conclusions1. Nothing special<jupyter_code>plot_countplot('dependents', 600)
cal_mean_val('dependents')<jupyter_output><empty_output><jupyter_text>Telephone Conclusions1. Nothing special<jupyter_code>plot_countplot('telephone', 420)
cal_mean_val('telephone')<jupyter_output><empty_output><jupyter_text>Present residence  ConclusionNothing special<jupyter_code>plot_countplot('present_residence', 300)
cal_mean_val('present_residence')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1260021865889213, 'fraction_numerical': 0.04163629737609329, 'mean_word_length': 6.198032786885246, 'pattern_counts': {'":': 7, '<': 127, '<?xml version=': 0, '>': 116, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '28', 'dir': 'jupyter-structured-clean-dedup', 'id': '636797', 'license': 'Apache-2.0', 'n_tokens_mistral': 4068, 'n_tokens_neox': 3592, 'n_words': 853, 'path': 'Projekty/Projekt1/Grupa1/JakubowskiKorbinSlapek/scripts/eda.ipynb', 'repo_name': 'niladrem/2020L-WUM', 'size': '10976'}