<jupyter_start><jupyter_text>**SVM**This code up is used to classify a dataset whose labels are a binary (either 1 and 0) a range of hyperparameters are tried. The best hyperparameters are used to calculate the Gini importance of each feature. The Gini importance allows us to remove features with a rank of 0, which means the model does not find it useful.**Note:** A uploaded dataset must have one column labeled "Label" with the row values as a 1 or 0 otherwise this code will not work<jupyter_code>""" Import libraries """
import matplotlib.pyplot as plt
import numpy as np
import io
import pandas as pd
import sklearn
import csv


""" Import sklearn specific functions """
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV
from sklearn.model_selection import StratifiedKFold
from sklearn.model_selection import cross_val_predict
from sklearn.metrics import f1_score
from sklearn.metrics import confusion_matrix
from sklearn.metrics import precision_score, recall_score
from sklearn.preprocessing import MinMaxScaler
from sklearn.pipeline import Pipeline

""" Import excel writer to allow excel notebooks to be written """
!pip install xlsxwriter
import xlsxwriter<jupyter_output>Collecting xlsxwriter
  Downloading XlsxWriter-1.4.5-py2.py3-none-any.whl (149 kB)
[K     |████████████████████████████████| 149 kB 8.3 MB/s 
[?25hInstalling collected packages: xlsxwriter
Successfully installed xlsxwriter-1.4.5<jupyter_text>The following code is used to upload as many files as you want to be classified<jupyter_code>from google.colab import files
uploaded = files.upload()<jupyter_output><empty_output><jupyter_text>The best measurements to use for classification are:* precision* recall* accuracy* f1* roc_aucBe careful with precision and recall as you can achieve 100% of one and have a really low other. F1 score is the harmoninc mean of precision and recall therefore should be used instread **calculate_cross_validation_value**Is used to fond the largest fold value possible for CV<jupyter_code>def calculate_cross_validation_value(y_train):
  """ aqurie the smallest number group of labels and return this for CV """
  arr = y_train.to_numpy()
  bin_arr = np.bincount(arr)
  return bin_arr.min()<jupyter_output><empty_output><jupyter_text>**grid_search_RF_function**This function is used to perfrom a grid search of defined hyperparametersOnce complete this model will the models best parameters<jupyter_code>def grid_search_function(x_train, y_train, Varible_list_1 , number_of_cv_folds):

  """ Initalise the SVM model with the best hyperparameters and setup the MinMaxscaler() """
  model = SVC(random_state=0)  
  min_max = MinMaxScaler()
  
  """ Setup the pipeline to perform scaling and model """
  pipe = Pipeline(steps = [
                               ("min_max" , min_max ),
                               
                               ("model" , model )
                               
  ])

  """ Set up the parameters for grid search with desired varibles for it to try """
  param_grid = [
        {'model__kernel': ['linear', 'rbf', 'poly'], 'model__C': Varible_list_1, }
  ]

  """ Assign the scoring type for grid search """
  scoring_type = 'f1'
  
  """ Set up the gridsearch using defined varible """
  grid_search = GridSearchCV(estimator= pipe, param_grid= param_grid,  cv= number_of_cv_folds , scoring=scoring_type,  return_train_score=True, n_jobs=-1) #n_jobs=-1 will use all avaliable processors avaliable

  """ Perfrom grid search with the x and y train data """
  grid_search.fit(X= x_train, y= y_train)

  """ Aquire the best score and round to 4 decimal places and print the best parameters"""
  best_accuracy = round(grid_search.best_score_,4)
  print("[INFO] Best score: " + str(best_accuracy) + " with parameters: "  + str(grid_search.best_params_) )

  """ Create a list of the best results from the Gridsearch """
  best_model_details = ([ "SVM",
                        scoring_type,
                        best_accuracy,
                        grid_search.best_params_['model__kernel'],
                        grid_search.best_params_['model__C'],
                        str(number_of_cv_folds.n_splits),
                      
  ])

  """ Resturn the best model results and the parameters that will be used for feature reduction """ 
  return best_model_details, grid_search.best_params_<jupyter_output><empty_output><jupyter_text>**graph_important_features** Is used to graph the importance of the featuresAs it is a function the values can be changed to make the graph clear.<jupyter_code>def graph_important_features(model, x_train_df):

  print("Feature importances of the input data obtained from (feature_importances_) for SVM")

  importance = np.abs(model.feature_importances_)
  feature_names = np.array(x_train_df.columns)

  f, ax = plt.subplots(figsize=(30,5))
  plt.bar(height=importance, x=feature_names, )

  plt.xticks(rotation='vertical', fontsize = 16)
  plt.yticks(fontsize = 16)

  plt.xlabel("Feature", fontsize = 20)
  plt.ylabel("Gini importance", fontsize = 20)

  plt.show()<jupyter_output><empty_output><jupyter_text>**best_model_test**THis functions uses the best hyperparameters and the reduced dataset to perform cross validation.This function then returns the results and are used as the final result<jupyter_code>def best_model_test(x_train, y_train, number_of_cv_folds, best_paramater):

  """ Initalise the SVM model with the best hyperparameters and setup the MinMaxscaler() """
  model = SVC(random_state=0, kernel = best_paramater['model__kernel'], C = best_paramater['model__C'])  
  min_max = MinMaxScaler()
  
  """ Setup the pipeline to perform scaling and model """
  pipe = Pipeline(steps = [
                               ("min_max" , min_max ),
                               
                               ("model" , model )
                               
  ])

  """ Use cross_val_predict to perform cross validation and get the predicted labels """
  y_train_pred = cross_val_predict(pipe, x_train, y_train, cv=number_of_cv_folds)

  """ Use the labels to produce a confusion matrix """
  conf = confusion_matrix(y_train, y_train_pred)
  print(conf)

  """ Calcualte the accuracies to .4dp which are then printed"""
  f1        = round(f1_score(y_train, y_train_pred), 4)
  precision = round(precision_score(y_train, y_train_pred), 4)
  recall    = round(recall_score(y_train, y_train_pred), 4)

  print("F1 score\tPurity\t\tRecovery")
  print(str(f1) + "\t\t" + str(precision) + "\t\t" + str(recall) + "\n\n")
    
  """ Write the actual labels and predicted labels to a list. Work out whether the preidction was correct, if so write correct. This is for easy reference """
  predictions_vs_acutal = []
  for index, row_data in enumerate(y_train):  
      if y_train[index] == y_train_pred[index]:
        correct_str = "Correct"
      else:
        correct_str = "Incorrect"
      predictions_vs_acutal.insert(index, [y_train[index], y_train_pred[index], correct_str]   )


  """ Create a list that has the accuracy results of the best model """
  model_check = ([str(f1), str(precision), str(recall), str(conf), str(len(x_train.columns)),  str(number_of_cv_folds.n_splits)])


  return predictions_vs_acutal, model_check<jupyter_output><empty_output><jupyter_text>Headers used for the exported CSV file<jupyter_code>def write_results_to_excel (file_name, grid_search_list, model_check, predictions_vs_acutal):

  """ Create headers used for the output file """
  header = ["Algorithm", "Scoring type", "Score", "kernel",  "C", "Stratified KFold splits"]
  model_check_header = ["F1 score", "Purity", "Recovery", "Confusion matrix", "Number of inputs", "Stratified KFold splits"]
  predicion_header = ["Label" , "Prediction", "Result"]

  """Create the workbook """
  workbook = xlsxwriter.Workbook(str(file_name) + '_svm.xlsx')
  worksheet = workbook.add_worksheet("SVM")

  """ Wrtie the Grid search results """
  worksheet.write(0, 0, "Grid search results")
  for row_num, row_data in enumerate(header):
    worksheet.write(1, row_num, row_data) 
  row = 2
  for row_num, row_data in enumerate(grid_search_list):
      for col_num, col_data in enumerate(row_data):
          worksheet.write(row, col_num, str(col_data))
      row+=1
  row+=1

  """ Write the best model results for final CV test with reduced dataset"""
  worksheet.write(row, 0, "Best model testing result")
  row+=1
  for row_num, row_data in enumerate(model_check_header):
    worksheet.write(row, row_num, row_data)
  row +=1
  for row_num, row_data in enumerate(model_check):
    worksheet.write(row, row_num, row_data) 
  row +=2

  """ Write the predicted values againest actual """
  worksheet.write(row, 0, "Prediction results for the best model")
  row +=1 
  for row_num, row_data in enumerate(predicion_header):
    worksheet.write(row, row_num, row_data) 
  row+=1

  for row_num, row_data in enumerate(predictions_vs_acutal):
      for col_num, col_data in enumerate(row_data):
          worksheet.write(row, col_num, str(col_data))
      row+=1

  """ Close the workboot and download"""
  workbook.close()
  files.download(str(file_name) + '_svm.xlsx') 

print("[INFO] CODE START")

""" Loop through all uploaded files"""
for file_name in uploaded:
    grid_search_list = []  
    reduction_mode = False

    """ Convert the uploaded .csv file to a dataframe """
    print("[INFO] File uploaded: " +str(file_name))
    loaded_file = pd.read_csv(io.BytesIO(uploaded[file_name]))

    """ Create the x and y train datasets"""
    x_train  = loaded_file.drop(['Label'], axis='columns')
    y_train     = loaded_file.Label

    """ Calculate the largest of number of fold for StratifiedKfold cross-vaildation"""
    number_of_cv_folds = StratifiedKFold(calculate_cross_validation_value(y_train)) #This function can be changed to a int if a dataset is large as it will take long to run, e.g. 10
    print("[INFO] Number of CV folds: " + str(number_of_cv_folds.n_splits))

    """ Perform Gird search for the best hyperparameters and appended results to a list"""
    print("[INFO] Finding best model parameters")
    grid_search_result, best_paramater = grid_search_function(x_train, y_train, [0.01, 0.1, 1, 10, 100], number_of_cv_folds)   
    grid_search_list.append(grid_search_result)

    """ Using the best parameters and reduced dataset aquire the accuracy results and preductions """
    print("[INFO] Final cross-fold valudation test using the best parameters")
    predictions_vs_acutal, model_check = best_model_test(x_train, y_train, number_of_cv_folds, best_paramater,)
    
    """ Write all the results of the .csv file to an excel work book """
    write_results_to_excel(file_name, grid_search_list, model_check, predictions_vs_acutal)<jupyter_output>[INFO] CODE START
[INFO] File uploaded: Aluminium_3-8Zor_dataset_updated.csv
[INFO] Number of CV folds: 13
[INFO] Finding best model parameters
[INFO] Best score: 0.8333 with parameters: {'model__C': 100, 'model__kernel': 'rbf'}
[INFO] Final cross-fold valudation test using the best parameters
[[33  4]
 [ 1 12]]
F1 score	Purity		Recovery
0.8276		0.75		0.9231
{'fraction_non_alphanumeric': 0.0980884860450507, 'fraction_numerical': 0.010589607825540697, 'mean_word_length': 4.093235831809872, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 27, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '117731', 'license': 'Apache-2.0', 'n_tokens_mistral': 3586, 'n_tokens_neox': 3414, 'n_words': 1071, 'path': 'SVM_assessment_with_comments.ipynb', 'repo_name': 'KOdin2/machine_learning', 'size': '11143'}