<jupyter_start><jupyter_text>Now lets use pandas to load the dataset from a url.<jupyter_code>import pandas
url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data'
dataset = pandas.read_csv(url)
print('shape=',dataset.shape)
print(dataset.head())<jupyter_output>shape= (149, 5)
   5.1  3.5  1.4  0.2  Iris-setosa
0  4.9  3.0  1.4  0.2  Iris-setosa
1  4.7  3.2  1.3  0.2  Iris-setosa
2  4.6  3.1  1.5  0.2  Iris-setosa
3  5.0  3.6  1.4  0.2  Iris-setosa
4  5.4  3.9  1.7  0.4  Iris-setosa<jupyter_text>Oh no, the dataset has no column names, lets add them<jupyter_code>dataset.columns=['A','B','C','D','E']
print(dataset.head())<jupyter_output>A    B    C    D            E
0  4.9  3.0  1.4  0.2  Iris-setosa
1  4.7  3.2  1.3  0.2  Iris-setosa
2  4.6  3.1  1.5  0.2  Iris-setosa
3  5.0  3.6  1.4  0.2  Iris-setosa
4  5.4  3.9  1.7  0.4  Iris-setosa<jupyter_text>Lets look at the class distribution of the column we will use as target for our machine learning tasks, column 'E'.<jupyter_code>print(dataset.groupby('E').size())<jupyter_output>E
Iris-setosa        49
Iris-versicolor    50
Iris-virginica     50
dtype: int64<jupyter_text>Now lets split our dataset randomly into training, testing, and validation datasets. First, shuffle the dataset randomly with replacement.<jupyter_code># The frac keyword argument specifies the fraction of rows to return in
# the random sample, so frac=1 means return all rows (in random order).
dataset_shuffled = dataset.sample(frac=1)
print(dataset_shuffled.head())<jupyter_output>A    B    C    D                E
9    5.4  3.7  1.5  0.2      Iris-setosa
113  5.8  2.8  5.1  2.4   Iris-virginica
96   6.2  2.9  4.3  1.3  Iris-versicolor
118  6.0  2.2  5.0  1.5   Iris-virginica
147  6.2  3.4  5.4  2.3   Iris-virginica<jupyter_text>Now split the data into training and validation sets in 80:20 ratio. We will use the validation set to test the performance of machine learning algorithms.<jupyter_code>n = dataset_shuffled.shape[0]
validation_size = int(0.2*n)

from sklearn import model_selection

data_array = dataset_shuffled.values
X = data_array[:,0:4]  # separate the predictor variables
Y = data_array[:,4]    # from the target variable (last column)
X_train, X_validation, Y_train, Y_validation = model_selection.train_test_split(X,Y,test_size=validation_size,random_state=13)<jupyter_output><empty_output><jupyter_text>Check the sizes of the training and validation data sets.<jupyter_code>print('Train_X = ',X_train.shape, 'Train_Y = ',Y_train.shape)
print('Val_X = ',X_validation.shape, 'Val_Y = ',Y_validation.shape)
import sklearn
from sklearn import linear_model
from sklearn import tree
from sklearn import svm
from sklearn import neighbors

# we will use these classifiers
LR = linear_model.LogisticRegression()
KNN = neighbors.KNeighborsClassifier()
DT = tree.DecisionTreeClassifier()
SVM = svm.SVC()

models_list = [LR,KNN,DT,SVM]
num_folds = 8
kfolds = model_selection.KFold(n_splits=num_folds, shuffle=True, random_state=13)
for model in models_list:
    model_name = type(model).__name__
    cv_results = model_selection.cross_val_score(model,X_train,Y_train,scoring='accuracy',cv=kfolds)
    print(model_name,' : ',cv_results.tolist())
    print(model_name,' : mean=',cv_results.mean(), ' sd=',cv_results.std())
    print('------------------------------------------------------------------------')<jupyter_output>LogisticRegression  :  [1.0, 1.0, 1.0, 0.6666666666666666, 0.9333333333333333, 0.9333333333333333, 0.9333333333333333, 1.0]
LogisticRegression  : mean= 0.9333333333333333  sd= 0.10540925533894599
------------------------------------------------------------------------
KNeighborsClassifier  :  [1.0, 0.9333333333333333, 1.0, 0.8666666666666667, 1.0, 0.9333333333333333, 1.0, 1.0]
KNeighborsClassifier  : mean= 0.9666666666666667  sd= 0.04714045207910316
------------------------------------------------------------------------
DecisionTreeClassifier  :  [0.9333333333333333, 0.9333333333333333, 0.9333333333333333, 0.9333333333333333, 1.0, 0.9333333333333333, 1.0, 0.9333333333333333]
DecisionTreeClassifier  : mean= 0.95  sd= 0.02886751345948128
------------------------------------------------------------------------
SVC  :  [0.9333333333333333, 1.0, 0.9333333333333333, 0.9333333333333333, 1.0, 0.9333333333333333, 1.0, 1.0]
SVC  : mean= 0.9666666666666667  sd= 0.033333333333333326
-------------[...]<jupyter_text>But this is naive training with each of the classifiers as we have not done anything to optimize the parameters for each classifer. We will use sklearn.model_selection.GridSearchCV for that. Lets do this with each classifier separately because each classifier will have its own set of parametes to tune.<jupyter_code>kfolds = model_selection.KFold(n_splits=num_folds,shuffle=True,random_state=13)

#define function to be called to do grid search for each of the classifiers.

def print_cv_results(estimator,results):
    means = results.cv_results_['mean_test_score']
    stds = results.cv_results_['std_test_score']
    print(type(estimator).__name__)
    for mean, std, params in zip(means, stds, results.cv_results_['params']):
        print("%0.3f (+/-%0.03f) for %r" % (mean, std * 2, params))
    print('best_parameters=',results.best_params_)
    print('best_accuracy=',results.best_score_)
    print('----------------------------------------------')
    
def do_grid_search(estimator,grid_values,kfolds):
    clf = model_selection.GridSearchCV(estimator=estimator,param_grid=grid_values,scoring='accuracy',cv=kfolds)
    results = clf.fit(X_train,Y_train)
    print_cv_results(estimator,results)
    return clf
# First, logistic Regression parameters to tune : C, penalty
grid_values = {'C':[0.01,0.1,1,10,100],'penalty':['l1','l2']}
clf_LR = do_grid_search(LR,grid_values,kfolds) 

# Next we do with KNN
grid_values = {'n_neighbors':list(range(1,10))}
clf_KNN = do_grid_search(KNN,grid_values,kfolds)

# Next Decision Tree
grid_values = {'criterion':['gini','entropy'],'splitter':['best','random'],'min_samples_split':list(range(2,10)),'min_samples_leaf':[1,2,3,4,5]}
clf_DT = do_grid_search(DT,grid_values,kfolds)
# And finally SVM
grid_values = {'C':[1,10,100],'kernel':['linear', 'poly', 'rbf', 'sigmoid'] }
clf_SVM = do_grid_search(SVM,grid_values,kfolds)
from sklearn.metrics import classification_report

# predictions for LR
Y_true, Y_pred = Y_validation, clf_LR.predict(X_validation)
print(classification_report(Y_true, Y_pred))
# predictions for KNN
Y_true, Y_pred = Y_validation, clf_KNN.predict(X_validation)
print(classification_report(Y_true, Y_pred))
# predictions for DT
Y_true, Y_pred = Y_validation, clf_DT.predict(X_validation)
print(classification_report(Y_true, Y_pred))
# predictions for SVM
Y_true, Y_pred = Y_validation, clf_SVM.predict(X_validation)
print(classification_report(Y_true, Y_pred))<jupyter_output>precision    recall  f1-score   support

    Iris-setosa       1.00      1.00      1.00        10
Iris-versicolor       0.90      1.00      0.95         9
 Iris-virginica       1.00      0.90      0.95        10

    avg / total       0.97      0.97      0.97        29
{'fraction_non_alphanumeric': 0.15923120089786758, 'fraction_numerical': 0.09918630751964086, 'mean_word_length': 3.500631313131313, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 21, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '126373', 'license': 'MIT', 'n_tokens_mistral': 2978, 'n_tokens_neox': 2589, 'n_words': 584, 'path': 'data/Python_simple_machine_learning_classification.ipynb', 'repo_name': 'pchanda/pchanda.github.io', 'size': '7128'}