<jupyter_start><jupyter_text>Indian-Liver-Patient-Classification______Dataset : https://archive.ics.uci.edu/ml/datasets/ILPD+(Indian+Liver+Patient+Dataset)This data set contains 10 variables that are age, gender, total Bilirubin, direct Bilirubin, total proteins, albumin, A/G ratio, SGPT, SGOT and Alkphos. Data Set Information:This data set contains 416 liver patient records and 167 non liver patient records.The data set was collected from north east of Andhra Pradesh, India. Selector is a class label used to divide into groups(liver patient or not). This data set contains 441 male patient records and 142 female patient records. Any patient whose age exceeded 89 is listed as being of age "90". Attribute Information:1. Age	Age of the patient 2. Gender	Gender of the patient 3. TB	Total Bilirubin 4. DB	Direct Bilirubin 5. Alkphos Alkaline Phosphotase 6. Sgpt Alamine Aminotransferase 7. Sgot Aspartate Aminotransferase 8. TP	Total Protiens 9. ALB	Albumin 10. A/G Ratio	Albumin and Globulin Ratio 11. Selector field used to split the data into two sets (labeled by the experts)<jupyter_code>import os
from sklearn.tree import DecisionTreeClassifier, export_graphviz
import pandas as pd
import numpy as np
from sklearn.cross_validation import train_test_split
from sklearn import cross_validation, metrics
from sklearn.ensemble import RandomForestClassifier
from sklearn.naive_bayes import BernoulliNB
from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC
from time import time
from sklearn import preprocessing
from sklearn.pipeline import Pipeline
from sklearn.metrics import roc_auc_score , classification_report
from sklearn.grid_search import GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.metrics import precision_score, recall_score, accuracy_score, classification_report

# read .csv from provided dataset
csv_filename="Indian Liver Patient Dataset (ILPD).csv"

# df=pd.read_csv(csv_filename,index_col=0)
df=pd.read_csv(csv_filename, 
              names=["Age", "Gender" , "TB" , "DB" , "Alkphos" , "Sgpt",
                     "Sgot" , "TP" , "ALB" , "A/G", "Selector" ])
df.head()
df.tail()
#Convert Gender,Selector to numbericals
le = preprocessing.LabelEncoder()
df['Gender'] = le.fit_transform(df.Gender)
 
df['Selector'] = le.fit_transform(df.Selector)
#Get binarized gender columns
#df['Gender'] = pd.get_dummies(df.Gender)
df.tail()
features=(list(df.columns[:-1]))
X = df[features]
y = df['Selector']
X.head()
"""
from sklearn.preprocessing import StandardScaler
# Scaling the features using StandardScaler:
X_scaler = StandardScaler()
y_scaler = StandardScaler()
X = X_scaler.fit_transform(X)
y = y_scaler.fit_transform(y)
X = X_scaler.transform(X)
y = y_scaler.transform(y)
"""
from sklearn.preprocessing import Imputer
X = Imputer().fit_transform(X)
# split dataset to 60% training and 40% testing
X_train, X_test, y_train, y_test = cross_validation.train_test_split(X, y, test_size=0.4, random_state=0)
print (X_train.shape, y_train.shape)<jupyter_output>(349, 10) (349,)<jupyter_text>Feature importances with forests of treesThis examples shows the use of forests of trees to evaluate the importance of features on an artificial classification task. The red bars are the feature importances of the forest, along with their inter-trees variability.<jupyter_code>%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

from sklearn.ensemble import ExtraTreesClassifier

# Build a classification task using 3 informative features

# Build a forest and compute the feature importances
forest = ExtraTreesClassifier(n_estimators=250,
                              random_state=0)

forest.fit(X, y)
importances = forest.feature_importances_
std = np.std([tree.feature_importances_ for tree in forest.estimators_],
             axis=0)
indices = np.argsort(importances)[::-1]

# Print the feature ranking
print("Feature ranking:")

for f in range(X.shape[1]):
    print("%d. feature %d - %s (%f) " % (f + 1, indices[f], features[indices[f]], importances[indices[f]]))

# Plot the feature importances of the forest
plt.figure(num=None, figsize=(14, 10), dpi=80, facecolor='w', edgecolor='k')
plt.title("Feature importances")
plt.bar(range(X.shape[1]), importances[indices],
       color="r", yerr=std[indices], align="center")
plt.xticks(range(X.shape[1]), indices)
plt.xlim([-1, X.shape[1]])
plt.show()
importances[indices[:5]]
for f in range(5):
    print("%d. feature %d - %s (%f)" % (f + 1, indices[f], features[indices[f]] ,importances[indices[f]]))
best_features = []
for i in indices[:5]:
    best_features.append(features[i])
# Plot the top 5 feature importances of the forest
plt.figure(num=None, figsize=(8, 6), dpi=80, facecolor='w', edgecolor='k')
plt.title("Feature importances")
plt.bar(range(5), importances[indices][:5], 
       color="r",  yerr=std[indices][:5], align="center")
plt.xticks(range(5), best_features)
plt.xlim([-1, 5])
plt.show()<jupyter_output><empty_output><jupyter_text>Decision Tree accuracy and time elapsed caculation<jupyter_code>t0=time()
print ("DecisionTree")

dt = DecisionTreeClassifier(min_samples_split=20,random_state=99)
# dt = DecisionTreeClassifier(min_samples_split=20,max_depth=5,random_state=99)

clf_dt=dt.fit(X_train,y_train)

print ("Acurracy: ", clf_dt.score(X_test,y_test))
t1=time()
print ("time elapsed: ", t1-t0)<jupyter_output>DecisionTree
Acurracy:  0.675213675214
time elapsed:  0.01000070571899414<jupyter_text>cross validation for DT<jupyter_code>tt0=time()
print ("cross result========")
scores = cross_validation.cross_val_score(dt, X, y, cv=3)
print (scores)
print (scores.mean())
tt1=time()
print ("time elapsed: ", tt1-tt0)<jupyter_output>cross result========
[ 0.64102564  0.67179487  0.60103627]
0.63795226075
time elapsed:  0.029001712799072266<jupyter_text>Tuning our hyperparameters using GridSearch<jupyter_code>from sklearn.metrics import classification_report

pipeline = Pipeline([
    ('clf', DecisionTreeClassifier(criterion='entropy'))
])

parameters = {
    'clf__max_depth': (5, 25 , 50),
    'clf__min_samples_split': (2, 5, 10),
    'clf__min_samples_leaf': (1, 2, 3)
}

grid_search = GridSearchCV(pipeline, parameters, n_jobs=-1, verbose=1, scoring='f1')
grid_search.fit(X_train, y_train)

print ('Best score: %0.3f' % grid_search.best_score_)
print ('Best parameters set:')

best_parameters = grid_search.best_estimator_.get_params()
for param_name in sorted(parameters.keys()):
    print ('\t%s: %r' % (param_name, best_parameters[param_name]))

predictions = grid_search.predict(X_test)

print (classification_report(y_test, predictions))<jupyter_output>Fitting 3 folds for each of 27 candidates, totalling 81 fits<jupyter_text>Random Forest accuracy and time elapsed caculation<jupyter_code>t2=time()
print ("RandomForest")
rf = RandomForestClassifier(n_estimators=100,n_jobs=-1)
clf_rf = rf.fit(X_train,y_train)
print ("Acurracy: ", clf_rf.score(X_test,y_test))
t3=time()
print ("time elapsed: ", t3-t2)<jupyter_output>RandomForest
Acurracy:  0.717948717949
time elapsed:  0.5140295028686523<jupyter_text>cross validation for RF<jupyter_code>tt2=time()
print ("cross result========")
scores = cross_validation.cross_val_score(rf, X, y, cv=3)
print (scores)
print (scores.mean())
tt3=time()
print ("time elapsed: ", tt3-tt2)<jupyter_output>cross result========
[ 0.7025641   0.68717949  0.72020725]
0.703316947877
time elapsed:  1.4800846576690674<jupyter_text>Tuning Models using GridSearch<jupyter_code>pipeline2 = Pipeline([
('clf', RandomForestClassifier(criterion='entropy'))
])

parameters = {
    'clf__n_estimators': (5, 25, 50, 100),
    'clf__max_depth': (5, 25 , 50),
    'clf__min_samples_split': (2, 5, 10),
    'clf__min_samples_leaf': (1, 2, 3)
}

grid_search = GridSearchCV(pipeline2, parameters, n_jobs=-1, verbose=1, scoring='accuracy', cv=3)

grid_search.fit(X_train, y_train)

print ('Best score: %0.3f' % grid_search.best_score_)

print ('Best parameters set:')
best_parameters = grid_search.best_estimator_.get_params()

for param_name in sorted(parameters.keys()):
    print ('\t%s: %r' % (param_name, best_parameters[param_name]))

predictions = grid_search.predict(X_test)
print ('Accuracy:', accuracy_score(y_test, predictions))
print (classification_report(y_test, predictions))<jupyter_output>Fitting 3 folds for each of 108 candidates, totalling 324 fits<jupyter_text>Naive Bayes accuracy and time elapsed caculation<jupyter_code>t4=time()
print ("NaiveBayes")
nb = BernoulliNB()
clf_nb=nb.fit(X_train,y_train)
print ("Acurracy: ", clf_nb.score(X_test,y_test))
t5=time()
print ("time elapsed: ", t5-t4)<jupyter_output>NaiveBayes
Acurracy:  0.713675213675
time elapsed:  0.07000398635864258<jupyter_text>cross-validation for NB<jupyter_code>tt4=time()
print ("cross result========")
scores = cross_validation.cross_val_score(nb, X,y, cv=3)
print (scores)
print (scores.mean())
tt5=time()
print ("time elapsed: ", tt5-tt4)<jupyter_output>cross result========
[ 0.71282051  0.71282051  0.71502591]
0.713555644126
time elapsed:  0.029001712799072266<jupyter_text>KNN accuracy and time elapsed caculation<jupyter_code>t6=time()
print ("KNN")
# knn = KNeighborsClassifier(n_neighbors=3)
knn = KNeighborsClassifier(n_neighbors=3)
clf_knn=knn.fit(X_train, y_train)
print ("Acurracy: ", clf_knn.score(X_test,y_test) )
t7=time()
print ("time elapsed: ", t7-t6)<jupyter_output>KNN
Acurracy:  0.679487179487
time elapsed:  0.011000633239746094<jupyter_text>cross validation for KNN<jupyter_code>tt6=time()
print ("cross result========")
scores = cross_validation.cross_val_score(knn, X,y, cv=5)
print (scores)
print (scores.mean())
tt7=time()
print ("time elapsed: ", tt7-tt6)<jupyter_output>cross result========
[ 0.66101695  0.64957265  0.63793103  0.57758621  0.72413793]
0.650048954228
time elapsed:  0.03300189971923828<jupyter_text>Fine tuning the model using GridSearch<jupyter_code>from sklearn.cross_validation import cross_val_score
from sklearn.pipeline import Pipeline
from sklearn import grid_search

knn = KNeighborsClassifier()

parameters = {'n_neighbors':[1,10]}

grid = grid_search.GridSearchCV(knn, parameters, n_jobs=-1, verbose=1, scoring='accuracy')


grid.fit(X_train, y_train)

print ('Best score: %0.3f' % grid.best_score_)

print ('Best parameters set:')
best_parameters = grid.best_estimator_.get_params()

for param_name in sorted(parameters.keys()):
    print ('\t%s: %r' % (param_name, best_parameters[param_name]))
    
predictions = grid.predict(X_test)
print( classification_report(y_test, predictions))<jupyter_output>Fitting 3 folds for each of 2 candidates, totalling 6 fits<jupyter_text>SVM accuracy and time elapsed caculation<jupyter_code>t7=time()
print ("SVM")

svc = SVC()
clf_svc=svc.fit(X_train, y_train)
print ("Acurracy: ", clf_svc.score(X_test,y_test) )
t8=time()
print ("time elapsed: ", t8-t7)<jupyter_output>SVM
Acurracy:  0.717948717949
time elapsed:  0.14200806617736816<jupyter_text>cross validation for SVM<jupyter_code>tt7=time()
print ("cross result========")
scores = cross_validation.cross_val_score(svc,X,y, cv=5)
print (scores)
print (scores.mean())
tt8=time()
print ("time elapsed: ", tt7-tt6)<jupyter_output>cross result========
[ 0.71186441  0.70940171  0.71551724  0.71551724  0.71551724]
0.713563568064
time elapsed:  47.82873582839966<jupyter_text>SVM with RBF kernel accuracy and time elapsed caculation<jupyter_code>t7=time()
print ("SVM")

svc_r = SVC(kernel='rbf')
clf_svc=svc.fit(X_train, y_train)
print ("Acurracy: ", clf_svc.score(X_test,y_test) )
t8=time()
print ("time elapsed: ", t8-t7)<jupyter_output>SVM
Acurracy:  0.717948717949
time elapsed:  0.03200173377990723<jupyter_text>cross validation for SVM<jupyter_code>tt7=time()
print ("cross result========")
scores = cross_validation.cross_val_score(svc_r,X,y, cv=5)
print (scores)
print (scores.mean())
tt8=time()
print ("time elapsed: ", tt7-tt6)<jupyter_output>cross result========
[ 0.71186441  0.70940171  0.71551724  0.71551724  0.71551724]
0.713563568064
time elapsed:  35752.70882368088
{'fraction_non_alphanumeric': 0.1289865933882921, 'fraction_numerical': 0.06661670413856274, 'mean_word_length': 6.5487115021998745, 'pattern_counts': {'":': 0, '<': 53, '<?xml version=': 0, '>': 53, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '390181', 'license': 'MIT', 'n_tokens_mistral': 4884, 'n_tokens_neox': 4313, 'n_words': 1017, 'path': 'Indian-Liver-Patient-Classification.ipynb', 'repo_name': 'Raavada/Applied-Machine-Learning', 'size': '12009'}