<jupyter_start><jupyter_text>Introducing the credit card defualt dataset  Data Set Information:**This research aimed at the case of customers default payments in Taiwan** Features description:- LIMIT_BAL: Amount of the given credit (NT dollar): it includes both the individual consumer credit and his/her family (supplementary) credit. - SEX: Gender (1 = male; 2 = female). - EDUCATION: Education (1 = graduate school; 2 = university; 3 = high school; 4 = others). - MARRIAGE: Marital status (1 = married; 2 = single; 3 = others). - AGE: Age (year). - PAY_1 - PAY_6: History of past payment. We tracked the past monthly payment records (from April to September, 2005) as follows: 1 = the repayment status in September, 2005; 1 = the repayment status in August, 2005; . . .; 6 = the repayment status in April, 2005. The measurement scale for the repayment status is: -1 = pay duly; 1 = payment delay for one month; 2 = payment delay for two months; . . .; 8 = payment delay for eight months; 9 = payment delay for nine months and above.- BILL_AMT1-BILL_AMT6: Amount of bill statement (NT dollar). X12 = amount of bill statement in September, 2005; X13 = amount of bill statement in August, 2005; . . .; X17 = amount of bill statement in April, 2005. - PAY_AMT1-PAY_AMT6: Amount of previous payment (NT dollar).- default payment next month: **positive class: default | negative class: pay**<jupyter_code>import numpy as np
import pandas as pd
import os
DATA_DIR = '../data'
FILE_NAME = 'credit_card_default.csv'
data_path = os.path.join(DATA_DIR, FILE_NAME)
ccd = pd.read_csv(data_path, index_col="ID")
ccd.head()
ccd.shape
ccd.rename(columns=lambda x: x.lower(), inplace=True)<jupyter_output><empty_output><jupyter_text>Numerical features<jupyter_code>bill_amt_features = ['bill_amt'+ str(i) for i in range(1,7)]
pay_amt_features = ['pay_amt'+ str(i) for i in range(1,7)]
numerical_features = ['limit_bal','age'] + bill_amt_features + pay_amt_features
ccd[['limit_bal','age']].describe()
ccd[bill_amt_features].describe().round()
ccd[pay_amt_features].describe().round()<jupyter_output><empty_output><jupyter_text>Encoding categorical features<jupyter_code>ccd['male'] = (ccd['sex'] == 1).astype('int')
ccd['male'].head(n=10)
ccd['male'].mean()
ccd['education'].value_counts(sort=False)
ccd['grad_school'] = (ccd['education'] == 1).astype('int')
ccd['university'] = (ccd['education'] == 2).astype('int')
ccd['high_school'] = (ccd['education'] == 3).astype('int')
ccd.loc[(ccd['grad_school']==0) & (ccd['university']==0) & (ccd['high_school']==0)]['education'].head()<jupyter_output><empty_output><jupyter_text>Low variance features<jupyter_code>ccd['marriage'].value_counts(sort=False)
ccd['single'] = (ccd['marriage'] == 2).astype('int')
ccd['marital_other'] = (ccd['marriage'] == 3).astype('int')
print("Proportion of singles: ", ccd['single'].mean())
print("Proportion of other marital status: ", ccd['marital_other'].mean())
ccd['married'] = (ccd['marriage'] == 1).astype('int')
print(ccd['married'].var())
print(ccd['single'].var())
(ccd['married'] == (1 - ccd['single'])).mean()<jupyter_output><empty_output><jupyter_text>A brief introduction to Feature Engineering<jupyter_code>ccd['pay_1'].value_counts().sort_index()
# fixing the pay_i features
pay_features= ['pay_' + str(i) for i in range(1,7)]
for x in pay_features:
    ccd.loc[ccd[x] <= 0, x] = 0
# producing delayed features
delayed_features = ['delayed_' + str(i) for i in range(1,7)]
for pay, delayed in zip(pay_features, delayed_features):
    ccd[delayed] = (ccd[pay] > 0).astype(int)
ccd[delayed_features].mean()
ccd['months_delayed'] = ccd[delayed_features].sum(axis=1)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16135186699373127, 'fraction_numerical': 0.022621967838648133, 'mean_word_length': 6.414141414141414, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 22, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '580143', 'license': 'MIT', 'n_tokens_mistral': 1336, 'n_tokens_neox': 1250, 'n_words': 335, 'path': 'Chapter02/ch2-2-credit-card-default.ipynb', 'repo_name': 'arifmudi/Hands-On-Predictive-Analytics-with-Python', 'size': '3669'}