<jupyter_start><jupyter_text>Financial Inclusion in Africa  Background InformationFinancial inclusion is the main obstacles to economic and human development in Africa as per studies and research conducted. Only 13.9% of the Adult population is estimated to have access to commercial bank accounts.Acces to a bank account is a contributor to long term economic growth  GOAlPredict which individuals in Kenya,Uganda,Tanzania and Rwanda are most likely to have or use a bank account.   In the process reveal insights into some of the key demographic factors that might drive individuals’ financial outcomes.  Importing essential modules<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import os
import platform
def set_seed(SEED=42):
    np.random.seed = SEED
    os.environ['PYTHONHASHSEED'] = str(SEED)
    
set_seed()
version = platform.python_version()
print("Python : ",version)
print("Pandas : ",pd.__version__)
print("NumPy : ",np.__version__)
print("Seaborn : ",sns.__version__)<jupyter_output>Python :  3.6.9
Pandas :  1.1.5
NumPy :  1.18.5
Seaborn :  0.11.0<jupyter_text>Getting our data<jupyter_code>train = pd.read_csv('datasets/Train_v2.csv')
test = pd.read_csv('datasets/Test_v2.csv')
variable = pd.read_csv('datasets/VariableDefinitions.csv')
Submission = pd.read_csv('datasets/SubmissionFile.csv')<jupyter_output><empty_output><jupyter_text>Visualize and understand our data<jupyter_code>train.head()
train.shape
variable
train.info()
train.bank_account.replace(['Yes','No'],[1,0],inplace = True)
train.head()<jupyter_output><empty_output><jupyter_text>Visualize Categorical values to establish patterns and insights<jupyter_code>a = sns.barplot(x="year",y="bank_account",data=train)
a = a.set_ylabel("bank_account")<jupyter_output><empty_output><jupyter_text>It seems that the probability of having a bank account for an individual in year 2018 is high as compared to other years<jupyter_code>b = sns.barplot(x="location_type",y="bank_account",data=train)
b = b.set_ylabel("bank_account")
c = sns.barplot(x="cellphone_access",y="bank_account",data=train)
c = c.set_ylabel("bank_account")
d = sns.barplot(x="gender_of_respondent",y="bank_account",data=train)
d = d.set_ylabel("bank_account")
d = sns.barplot(x="relationship_with_head",y="bank_account",data=train)
d = d.set_ylabel("bank_account")<jupyter_output><empty_output><jupyter_text>This indicates that the elderly are less likely to have a bank account.It is also clear that if a respondent is the Head of the house, it is more likely for them to have a bank account<jupyter_code>plt.figure(figsize=(16,8))
d = sns.barplot(x="marital_status",y="bank_account",data=train)
d = d.set_ylabel("bank_account")<jupyter_output><empty_output><jupyter_text>Married individuals and those who are not aware where they stand in their relationship are more likely to have a bank account.My assumption is individuals who are not aware of their marital status are most likely to be youths already working and trying to settle<jupyter_code>plt.figure(figsize=(16,8))
d = sns.barplot(x="education_level",y="bank_account",data=train)
d = d.set_ylabel("bank_account")<jupyter_output><empty_output><jupyter_text>Individuals in tertiary and vocational training are most likely to have a bank account<jupyter_code>plt.figure(figsize=(16,8))
d = sns.barplot(x="job_type",y="bank_account",data=train)
d.set_xticklabels(d.get_xticklabels(), rotation=45, horizontalalignment='right')
d = d.set_ylabel("bank_account")<jupyter_output><empty_output><jupyter_text>Individuals sith formal employment are more likely to have a bank account. In summary we can say that an individual in formal employment, Who is the head of the house hold,with a vocation or tertially education, Married or about to get married(Dont Know-- likely in their prime youthful days),urban resident, with phone access, and surveyed in the year 2018 has 95 percent chance of having a bank account  Feature EngineeringWe concatenate the two datasets to achieve uniform feature conversions during thid exercise<jupyter_code>train_len = len(train)
train_test = pd.concat([train,test],axis=0).reset_index(drop=True)
train_test.info()
# Descriptive Analytics
tot_len = len(train_test.columns.tolist())
print('columns present are \n {} \n No of columns is {}'.format(train_test.columns.tolist(),tot_len))
train_test.count() # total number of values per column
leng = len(train_test.describe().keys().tolist())
print("No of numeric features is ", leng)
print("Number of non-numeric features are: ", tot_len-leng)
# visualize unique values in all the categorical features
n = train_test.nunique(axis=0)
print("unique values in each column are: \n", n)<jupyter_output>unique values in each column are: 
 country                       4
year                          3
uniqueid                  12480
bank_account                  2
location_type                 2
cellphone_access              2
household_size               20
age_of_respondent            85
gender_of_respondent          2
relationship_with_head        6
marital_status                5
education_level               6
job_type                     10
dtype: int64<jupyter_text>Categorical features that can be encoded are    * country    * year    * bank_account    * cellphone_access    * gender_of_respondent    * location_type<jupyter_code># Getting Dummies
def getdummies(dataset,col,pref):
    dataset = pd.get_dummies(dataset, columns = col, prefix=pref)
    return dataset
from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()

def col_trans(data,col):
  data[col] = le.fit_transform(data[col])

#train_test = getdummies(train_test,["cellphone_access","location_type","gender_of_respondent"],["ce","l","g"])
col_trans(train_test,"cellphone_access")
col_trans(train_test,"location_type")
col_trans(train_test,"gender_of_respondent")
train_test = getdummies(train_test,["country"],["co"])
train_test.head()
# lets drop the uniqueid since its insignificant
train_test.drop(columns=["uniqueid"],inplace=True)
def unique_vals(col):
    return pd.unique(train_test[col])
# Unique columns in ralationship column
print("relationship with head unique vals",unique_vals("relationship_with_head"))
print("marital_status unique vals",unique_vals("marital_status"))
print("education_level unique vals",unique_vals("education_level"))
print("job_type unique vals",unique_vals("job_type"))
# Transforming the relationship column
def col_transform(dataset,col,vals,reps): 
        dataset[col].replace(vals,reps,inplace = True)
col_transform(train_test,'relationship_with_head',['Spouse','Head of Household','Other relative','Child','Parent','Other non-relatives'],['S','H','O_R','C','P','N_R'])
# Transform marital_status column
col_transform(train_test,'marital_status',['Married/Living together','Widowed','Single/Never Married','Divorced/Seperated','Dont know'],['M','W','S','D','DN'])
# Transform education_level column
col_transform(train_test,'education_level',['Secondary education','No formal education','Vocational/Specialised training','Primary education','Tertiary education','Other/Dont know/RTA'],['S','N','V','P','T','ODR'])
# Transform job_type column
col_transform(train_test,'job_type',['Self employed', 'Government Dependent',
       'Formally employed Private', 'Informally employed',
       'Formally employed Government', 'Farming and Fishing',
       'Remittance Dependent', 'Other Income',
       'Dont Know/Refuse to answer', 'No Income'],['SE','GD','FEP','IE','FEG','FF','RD','OI','DK','NI'])

# Get dummies of the remaining categorical features
train_test = getdummies(train_test,["relationship_with_head","marital_status","education_level","job_type"],["r","m","e","j"])
train_test.head()<jupyter_output><empty_output><jupyter_text>Modelling Models to test    SVC    Decision Tree    AdaBoost    Random Forest    KNN    Logistic regression Other Models That can be used    Extra Trees    Gradient Boosting    Multiple layer perceprton (neural network)    Linear Discriminant Analysis<jupyter_code>train_test.info()
# obtain back our train and test datasets
train = train_test[:train_len]
testf = train_test[train_len:]
testf.drop(labels=["bank_account"],axis = 1,inplace=True)
## Separate train features and label 

#train["bank_account"] = train["bank_account"].astype(int)

Y_train = train["bank_account"]

X_train = train.drop(labels = ["bank_account"],axis = 1)
from sklearn.model_selection import GridSearchCV, cross_val_score, StratifiedKFold,train_test_split
from sklearn.metrics import confusion_matrix,accuracy_score #precision_score, recall_score,f1_score,
from xgboost import XGBClassifier
from sklearn.svm import SVC
from sklearn.ensemble import VotingClassifier,GradientBoostingClassifier

sns.set(style='white', context='notebook', palette='deep')<jupyter_output><empty_output><jupyter_text>Evaluating our model with    * Without Dimension Reduction  Lets try and Rescale our data to see its effecton overall perfomance<jupyter_code># minimal error before this action is 0.113..Random Forest
from sklearn.preprocessing import MinMaxScaler  
scaler = MinMaxScaler(feature_range=(0, 1))  

X_train_rescaled = scaler.fit_transform(X_train)
testf_rescaled = scaler.fit_transform(testf)
xtrain, xtest, ytrain, ytest = train_test_split(X_train_rescaled, Y_train, train_size=0.85,test_size=0.15, random_state=42)<jupyter_output><empty_output><jupyter_text>We evaluate model perfomance using gridsearchcv to optimize our XGBClassifier    * Finding the best hyper parameters<jupyter_code>kfold = StratifiedKFold(n_splits=10)

'''GBC = GradientBoostingClassifier()
gb_param_grid = {'loss' : ["deviance"],
              'n_estimators' : [100,200,300],
              'learning_rate': [0.1, 0.05, 0.01],
              'max_depth': [4, 8],
              'min_samples_leaf': [100,150],
              'max_features': [0.3, 0.1] 
              }

gsGBC = GridSearchCV(GBC,param_grid = gb_param_grid, cv=kfold, scoring="accuracy", n_jobs= 4, verbose = 10)

gsGBC.fit(xtrain,ytrain)

GBC_best = gsGBC.best_estimator_

# Best score
gsGBC.best_score_'''
#gsGBC.best_params_
#{'learning_rate': 0.05,'loss': 'deviance', 'max_depth': 8, 'max_features': 0.1, 'min_samples_leaf': 150, 'n_estimators': 300}
'''xgb_pipeline =XGBClassifier()
    
# param to be tested 
gbm_param_grid = {'min_child_weight': [1, 5, 7,10],
        'gamma': [0.5, 1, 1.5, 2, 5],
        'subsample': [0.6, 0.8, 1.0],
        'colsample_bytree': [0.6, 0.8, 1.0],
        'max_depth': [3, 4, 5,6,7]
        }

# configure the gridsearch 
grid_search = GridSearchCV(estimator=xgb_pipeline, param_grid=gbm_param_grid,n_jobs = -1, verbose=10 )
# n_jobs : number of thread in parallel if -1 means max thread used
# cv : number of fold in cross-validation

# train the models 
grid_search.fit(xtrain,ytrain)'''
#grid_search.best_params_

model1 = XGBClassifier(min_child_weight = 5, subsample = 1.0, colsample_bytree = 1.0, max_depth = 5,gamma=1)# min_child_weight = 5, subsample = 1.0, colsample_bytree = 1.0, max_depth = 5,gamma=1
results = cross_val_score(model1, xtrain, y = ytrain, scoring = "accuracy",cv = kfold, n_jobs=4)
error = 1 - results.mean()
print(error)
# 0.11232788894447232
model2 = GradientBoostingClassifier(learning_rate= 0.05,loss='deviance', max_depth= 8, max_features= 0.1, min_samples_leaf= 150, n_estimators= 300)
results = cross_val_score(model2, xtrain, y = ytrain, scoring = "accuracy",cv = kfold, n_jobs=4)
error = 1 - results.mean()
print(error)
#0.11212803901950985
model1.fit(xtrain,ytrain)
y_pred = model1.predict(xtest)

# Get error rate
print("Error rate of Random Forest classifier: ", 1 - accuracy_score(ytest, y_pred))

# Get confusion matrix
confusion_matrix(y_pred, ytest)
model2.fit(xtrain,ytrain)
y_pred = model2.predict(xtest)

# Get error rate
print("Error rate of Random Forest classifier: ", 1 - accuracy_score(ytest, y_pred))

# Get confusion matrix
confusion_matrix(y_pred, ytest)
# 0.11079625956361572
votingC = VotingClassifier(estimators=[('GBC', model2), ('XGB',model1)], voting='soft', n_jobs=-1)

votingC = votingC.fit(xtrain, ytrain)<jupyter_output><empty_output><jupyter_text>Summary<jupyter_code>Submission.head()
test.shape
final_results = list(map(float,votingC.predict(testf_rescaled)))
Submission['Bank_account'] = final_results
Submission['uniqueid'] = test['uniqueid'].astype(str) + ' x ' +  test['country'].astype(str)
Submission = Submission[['uniqueid','Bank_account']]
Submission.Bank_account.value_counts()
Submission.to_csv('ensemble1.csv', index=False)<jupyter_output><empty_output><jupyter_text>TODO Remodel with dimensionality reduction --done Remodel with additional feature addition Remodel with optimal k value for the knn classifier -- done Remodel with minmaxscaling --done remodel with country and year dropped - done Use the XGBoost model model with optimised svc hyperparameters  Findings1* When our data is rescalled, the svc perfoms better than The LogisticRegression resulting to minimal misclassification error.* Model perfomance 1. SVC 2. Logisticegression 3. KNN* It is clear that svc perfomes better with data of uniform size.  Findings 2accuracy before dimensionality reduction - 0.11262262916939181   accuracy after dimension reduction - 0.11556572923479413Logistic regression perfoms better than SVCDimensionality reduction desnt improve model perfomance in this case  Findings 3- optimal k value.21best error - 0.11262262916939181   error after test - 0.11576193590582073perfomance best than the previous KNN without optimised k value but still dont beat the svc  Findings 4 - optimised SVCsvc error before optimisation - 0.11262262916939181 error after grid search optimisation -0.10991012873451544  time of grid search optimisation - recommend running this in colab Grid searchCv optimises the SVC model and doesnt result to better perfomance..no idea why..sth must have been wrongly implemented  xGboost perfomance looks promising.. 0.107 and 0.109 is the difference in perfomance of first submission and my submission. TODO- clean notebook and find a way to improve it. Drop columns country and year -X    * find best learning rate...0.3    * Tweak the parameters further and understand the model better,..push it to the edge.GOal - beat 0.10954578576633467<jupyter_code>ten = pd.read_csv('financial_inclusionv10.csv')
elev = pd.read_csv('financial_inclusionv11.csv')
twel = pd.read_csv('financial_inclusionv12.csv')
ten.Bank_account.value_counts() # 0.112433075550268

elev.Bank_account.value_counts() # 0.110450128891533
twel.Bank_account.value_counts() # 0.1114416022209

#0.0    9396
#1.0     690
# v13 - 0.108665476898671

#v14 - 0.109656950228039

#0.0    9406
#1.0     680
#0.0    9400
#1.0     686
# v15 - 0.1114416022209
#0.0    9359
#1.0     727
# v16 - 0.110053539559786

#0.0    9367
#1.0     719
# v17 - 0.108467182232798 year column is not scaled to 2,1,0
#0.0    9349
#1.0     737
# v18 - 0.110251834225659

#0.0    9359
#1.0     727
# v19 - 0.110053539559786
#0.0    9385
#1.0     701
# v20 - 0.108467182232798 -- my params
#0.0    9344
#1.0     742
# v21 - 0.108863771564545 --git params
#0    9340
#1     746
# v22 - local notbbk -0.11084671822328

#btw - 9367 and 9385???

#0.0    9495
#1.0     591
# ens - 0.110648423557406
#0.0    9347
#1.0     739
#ens1 - 0.102..

#0 - 9367
# -
# 0.107674003569304<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10998318893055735, 'fraction_numerical': 0.05295486874434243, 'mean_word_length': 5.194233079695635, 'pattern_counts': {'":': 0, '<': 59, '<?xml version=': 0, '>': 59, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '30540', 'license': 'MIT', 'n_tokens_mistral': 5663, 'n_tokens_neox': 5091, 'n_words': 1314, 'path': 'Financial_Inclusion/ensemble1076.ipynb', 'repo_name': 'tevish321/DataScience', 'size': '15466'}