<jupyter_start><jupyter_text>Prerequisites Importing the libraries, set the column names and use pandas to load the dataset<jupyter_code>%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
cols = ['label','lepton pT', 'lepton eta', 'lepton phi', 'missing energy magnitude', 'missing energy phi', 'jet 1 pt', 'jet 1 eta', 'jet 1 phi', 'jet 1 b-tag', 'jet 2 pt', 'jet 2 eta', 'jet 2 phi', 'jet 2 b-tag', 'jet 3 pt', 'jet 3 eta', 'jet 3 phi', 'jet 3 b-tag', 'jet 4 pt', 'jet 4 eta', 'jet 4 phi', 'jet 4 b-tag', 'm_jj','m_jjj', 'm_lv', 'm_jlv', 'm_bb', 'm_wbb', 'm_wwbb']
df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/00280/HIGGS.csv.gz', compression='gzip', names=cols)<jupyter_output><empty_output><jupyter_text>`df.head()` will show us the first 5 rows of the imported dataset.<jupyter_code>df.head()<jupyter_output><empty_output><jupyter_text>`df.info()` will show a quick summary of the data.<jupyter_code>df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 11000000 entries, 0 to 10999999
Data columns (total 29 columns):
 #   Column                    Dtype  
---  ------                    -----  
 0   label                     float64
 1   lepton pT                 float64
 2   lepton eta                float64
 3   lepton phi                float64
 4   missing energy magnitude  float64
 5   missing energy phi        float64
 6   jet 1 pt                  float64
 7   jet 1 eta                 float64
 8   jet 1 phi                 float64
 9   jet 1 b-tag               float64
 10  jet 2 pt                  float64
 11  jet 2 eta                 float64
 12  jet 2 phi                 float64
 13  jet 2 b-tag               float64
 14  jet 3 pt                  float64
 15  jet 3 eta                 float64
 16  jet 3 phi                 float64
 17  jet 3 b-tag               float64
 18  jet 4 pt                  float64
 19  jet 4 eta                 float64
 20  jet 4 phi             [...]<jupyter_text>`df.describe()` will show as some of the data and their statistical properties.<jupyter_code>df.describe()<jupyter_output><empty_output><jupyter_text>Test set creation We pick 0.5% of the dataset to create a train set. Lets set the `random_state` to 20 for testing purposes.We will use the method `train_test_split()` from the library `scikit-learn`<jupyter_code>from sklearn.model_selection import train_test_split


train_set, test_set = train_test_split(df, test_size=.995, random_state=20)
test_set.head()<jupyter_output><empty_output><jupyter_text>Copying the training set.<jupyter_code>df = train_set.copy()<jupyter_output><empty_output><jupyter_text>Lets calculate the correlation.<jupyter_code>corr_matrix = df.corr()
corr_matrix['label'].sort_values(ascending=False)
corr_matrix<jupyter_output><empty_output><jupyter_text>Data Preparation<jupyter_code>predictors = train_set.drop('label', axis=1)
labels = train_set['label'].copy()
predictors.head()<jupyter_output><empty_output><jupyter_text>Since the dataset is massive we will create keep only some of the entries from the test set<jupyter_code>test_predictors = test_set.drop('label', axis=1)[:400000]
test_labels = test_set['label'].copy()[:400000]
test_predictors.head()<jupyter_output><empty_output><jupyter_text>Predictors<jupyter_code>predictors.info()
test_predictors.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 400000 entries, 808665 to 5539827
Data columns (total 28 columns):
 #   Column                    Non-Null Count   Dtype  
---  ------                    --------------   -----  
 0   lepton pT                 400000 non-null  float64
 1   lepton eta                400000 non-null  float64
 2   lepton phi                400000 non-null  float64
 3   missing energy magnitude  400000 non-null  float64
 4   missing energy phi        400000 non-null  float64
 5   jet 1 pt                  400000 non-null  float64
 6   jet 1 eta                 400000 non-null  float64
 7   jet 1 phi                 400000 non-null  float64
 8   jet 1 b-tag               400000 non-null  float64
 9   jet 2 pt                  400000 non-null  float64
 10  jet 2 eta                 400000 non-null  float64
 11  jet 2 phi                 400000 non-null  float64
 12  jet 2 b-tag               400000 non-null  float64
 13  jet 3 pt                  400000 non-n[...]<jupyter_text>We will now import all the necessary libraries and functions<jupyter_code>from sklearn.metrics import classification_report, confusion_matrix
from sklearn.tree import DecisionTreeClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC
from sklearn.linear_model import LogisticRegression<jupyter_output><empty_output><jupyter_text>Decision Tree Classifier<jupyter_code>classifier = DecisionTreeClassifier()
classifier.fit(predictors,labels)
y_pred = classifier.predict(predictors)
print(confusion_matrix(labels, y_pred))
print(classification_report(labels, y_pred))
print('Accuracy of Decision Tree classifier on traing set: {:.2f}'
     .format(classifier.score(predictors,labels)))
score_dectree=classifier.score(test_predictors,test_labels)
print('Accuracy of Decision Tree classifier on test set: {:.2f}'
     .format(score_dectree))<jupyter_output>[[25810     0]
 [    0 29190]]
              precision    recall  f1-score   support

         0.0       1.00      1.00      1.00     25810
         1.0       1.00      1.00      1.00     29190

    accuracy                           1.00     55000
   macro avg       1.00      1.00      1.00     55000
weighted avg       1.00      1.00      1.00     55000

Accuracy of Decision Tree classifier on traing set: 1.00
Accuracy of Decision Tree classifier on test set: 0.63<jupyter_text>Naive Bayes<jupyter_code>classifier = GaussianNB()
classifier.fit(predictors,labels)
y_pred = classifier.predict(predictors)
print(confusion_matrix(labels, y_pred))
print(classification_report(labels, y_pred))
print('Accuracy of GNB classifier on training set: {:.2f}'
     .format(classifier.score(predictors,labels)))
score_gnb=classifier.score(test_predictors,test_labels)
print('Accuracy of GNB classifier on test set: {:.2f}'
     .format(score_gnb))<jupyter_output>[[ 8338 17472]
 [ 4726 24464]]
              precision    recall  f1-score   support

         0.0       0.64      0.32      0.43     25810
         1.0       0.58      0.84      0.69     29190

    accuracy                           0.60     55000
   macro avg       0.61      0.58      0.56     55000
weighted avg       0.61      0.60      0.57     55000

Accuracy of GNB classifier on training set: 0.60
Accuracy of GNB classifier on test set: 0.59<jupyter_text>Logistic Regression<jupyter_code>classifier = LogisticRegression()
classifier.fit(predictors,labels)
y_pred = classifier.predict(predictors)
print(confusion_matrix(labels, y_pred))
print(classification_report(labels, y_pred))
print('Accuracy of Logistic Regression classifier on training set: {:.2f}'
     .format(classifier.score(predictors, labels)))
score_logreg=classifier.score(test_predictors, test_labels)
print('Accuracy of Logistic Regression classifier on test set: {:.2f}'
     .format(score_logreg))<jupyter_output>[[13614 12196]
 [ 7413 21777]]
              precision    recall  f1-score   support

         0.0       0.65      0.53      0.58     25810
         1.0       0.64      0.75      0.69     29190

    accuracy                           0.64     55000
   macro avg       0.64      0.64      0.64     55000
weighted avg       0.64      0.64      0.64     55000

Accuracy of Logistic Regression classifier on training set: 0.64
Accuracy of Logistic Regression classifier on test set: 0.64<jupyter_text>K nearest neighbors<jupyter_code>classifier = KNeighborsClassifier(n_neighbors=3)
classifier.fit(predictors,labels)
y_pred = classifier.predict(predictors)
print(confusion_matrix(labels, y_pred))
print(classification_report(labels, y_pred))
print('Accuracy of K-NN classifier on training set: {:.2f}'
     .format(classifier.score(predictors, labels)))
score_knn=classifier.score(test_predictors, test_labels)
print('Accuracy of K-NN classifier on test set: {:.2f}'
     .format(score_knn))<jupyter_output>[[18965  6845]
 [ 4798 24392]]
              precision    recall  f1-score   support

         0.0       0.80      0.73      0.77     25810
         1.0       0.78      0.84      0.81     29190

    accuracy                           0.79     55000
   macro avg       0.79      0.79      0.79     55000
weighted avg       0.79      0.79      0.79     55000

Accuracy of K-NN classifier on training set: 0.79
Accuracy of K-NN classifier on test set: 0.58<jupyter_text>Support Vector Machines<jupyter_code>classifier = SVC()
classifier.fit(predictors,labels)
y_pred = classifier.predict(predictors)
print(confusion_matrix(labels, y_pred))
print(classification_report(labels, y_pred))
print('Accuracy of SVM classifier on training set: {:.2f}'
     .format(classifier.score(predictors,labels)))
score_svm=classifier.score(test_predictors,test_labels)
print('Accuracy of SVM classifier on test set: {:.2f}'
     .format(score_svm))<jupyter_output>[[14997 10813]
 [ 5525 23665]]
              precision    recall  f1-score   support

         0.0       0.73      0.58      0.65     25810
         1.0       0.69      0.81      0.74     29190

    accuracy                           0.70     55000
   macro avg       0.71      0.70      0.70     55000
weighted avg       0.71      0.70      0.70     55000

Accuracy of SVM classifier on training set: 0.70
Accuracy of SVM classifier on test set: 0.67<jupyter_text>Results We will now plot the scores of each classifier on the test set<jupyter_code>fig = plt.figure()
ax = fig.add_axes([0,0,1,1])
classifiers = ['Decission Tree', 'Naive Bayes', 'Logistic Regression', 'K-NN', 'SVM']
scores = [score_dectree,score_gnb,score_logreg,score_knn,score_svm]
ax.set_ylabel('Scores')
ax.set_title('Classifier')
ax.bar(classifiers,scores)

plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09238699444885012, 'fraction_numerical': 0.07801348136399683, 'mean_word_length': 2.658085569253082, 'pattern_counts': {'":': 0, '<': 64, '<?xml version=': 0, '>': 64, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '597731', 'license': 'MIT', 'n_tokens_mistral': 3826, 'n_tokens_neox': 3244, 'n_words': 892, 'path': 'Data Analysis/project2/ergasia2_e18089.ipynb', 'repo_name': 'dim1119/university-assignments', 'size': '10088'}