<jupyter_start><jupyter_text>Support Vector Machine Classifier (SVM, SVC) Support vector machines are supervised learning models with associated learning algorithms that analyze data used for classification and regression analysis.<jupyter_code>%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from sklearn.svm import SVC # "Support vector classifier"
# model = SVC(kernel='poly', C=1E10)
def plot_svc_decision_function(model, ax=None, plot_support=True):
    """Plot the decision function for a 2D SVC"""
    if ax is None:
        ax = plt.gca()
    xlim = ax.get_xlim()
    ylim = ax.get_ylim()
    
    # create grid to evaluate model
    x = np.linspace(xlim[0], xlim[1], 30)
    y = np.linspace(ylim[0], ylim[1], 30)
    Y, X = np.meshgrid(y, x)
    xy = np.vstack([X.ravel(), Y.ravel()]).T
    P = model.decision_function(xy).reshape(X.shape)
    
    # plot decision boundary and margins
    ax.contour(X, Y, P, colors='k',
               levels=[-1, 0, 1], alpha=0.5,
               linestyles=['--', '-', '--'])
    
    # plot support vectors
    if plot_support:
        ax.scatter(model.support_vectors_[:, 0],
                   model.support_vectors_[:, 1],
                   s=300, linewidth=1, facecolors='none');
    ax.set_xlim(xlim)
    ax.set_ylim(ylim)

from sklearn.datasets.samples_generator import make_circles
X, y = make_circles(100, factor=.1, noise=.1)



from sklearn.model_selection import train_test_split
import pandas as pd
import numpy as np

from sklearn.datasets.samples_generator import make_blobs
X, y = make_blobs(n_samples=300, centers=2, n_features=2,random_state=0)


X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# clf = SVC(kernel='sigmoid').fit(X_train, y_train)
clf = SVC(kernel='linear').fit(X_train, y_train)

#plt.scatter(X_train[:, 0], X_train[:, 1], c=y_train, s=50, cmap='autumn')
plt.scatter(X_test[:, 0], X_test[:, 1], c=y_test, s=50, cmap='winter')

plot_svc_decision_function(clf, plot_support=False);

# print(len(y))
# print(len(X))

# print(X[10])
# print(y[10])

print(y_test)
print(clf.predict(X_test))


df = pd.DataFrame()
df['predicted'] = clf.predict(X_test).astype(int)
df['labels'] = y_test.astype(int)

df['correct'] = np.where((df['predicted'] == df['labels']), True, False)

df
# df[df.a > 1].sum()<jupyter_output><empty_output><jupyter_text>Machine Learning metrics  Classification Accuracy Classification Accuracy is what we usually mean, when we use the term accuracy. It is the ratio of number of correct predictions to the total number of input samples.$$Accuracy=\frac{Number\:of\:Correct\:predictions}{Total\:number\:of\:predictions\:made}$$It works well only if there are equal number of samples belonging to each class.<jupyter_code>### PUT HERE CODE THAT COMPUTES ACCURACY<jupyter_output><empty_output><jupyter_text>Confusion Matrix Confusion Matrix as the name suggests gives us a matrix as output and describes the complete performance of the model.Lets assume we have a binary classification problem. We have some samples belonging to two classes : YES or NO. Also, we have our own classifier which predicts a class for a given input sample. On testing our model on 165 samples ,we get the following result.n=165    |       -         |   Predicted: NO   |   Predicted: YES  ||-----------------|-------------------|-------------------||   Actual  NO    |         50        |        10         ||   Actual  YES   |         5         |       100         |$$Confusion Matrix$$There are 4 important terms :* True Positives : The cases in which we predicted YES and the actual output was also YES.* True Negatives : The cases in which we predicted NO and the actual output was NO.* False Positives : The cases in which we predicted YES and the actual output was NO.* False Negatives : The cases in which we predicted NO and the actual output was YES.Accuracy for the matrix can be calculated by taking average of the values lying across the “main diagonal” i.e$$Accuracy=\frac{True\:Positives+False\:Negatives}{Total\:Number\:of\:Samples}$$Confusion Matrix forms the basis for the other types of metrics.<jupyter_code>### PUT HERE CODE THAT SHOWS CONFUSION MATRIX<jupyter_output><empty_output><jupyter_text>F1 Score F1 Score is used to measure a test’s accuracyF1 Score is the Harmonic Mean between precision and recall. The range for F1 Score is [0, 1]. It tells you how precise your classifier is (how many instances it classifies correctly), as well as how robust it is (it does not miss a significant number of instances).High precision but lower recall, gives you an extremely accurate, but it then misses a large number of instances that are difficult to classify. The greater the F1 Score, the better is the performance of our model. Mathematically, it can be expressed as :$$F1=2\times\frac{Precision\times Recall}{Precision+Recall}$$F1 Score tries to find the balance between precision and recall.* Precision : It is the number of correct positive results divided by the number of positive results predicted by the classifier.$$Precision=\frac{True\:Positives}{True\:Positives+False\:Positives}$$* Recall : It is the number of correct positive results divided by the number of all relevant samples (all samples that should have been identified as positive).$$Recall=\frac{True\:Positives}{True\:Positives+False\:Negatives}$$<jupyter_code>### CLASS PRACTICAL TASK 1
### INSERT HERE COMPUTATIONS FOR: ACC RC RP F1
### Do it from scratch and using scikit-learn api


### END YOUR BLOCK
from sklearn.datasets import make_moons
from matplotlib import pyplot
from pandas import DataFrame
# generate 2d classification dataset
X, y = make_moons(n_samples=100, noise=0.1)
# scatter plot, dots colored by class value
df = DataFrame(dict(x=X[:,0], y=X[:,1], label=y))
colors = {0:'red', 1:'blue'}
fig, ax = pyplot.subplots()
grouped = df.groupby('label')
for key, group in grouped:
    group.plot(ax=ax, kind='scatter', x='x', y='y', label=key, color=colors[key])
pyplot.show()
### CLASS PRACTICAL TASK 2
### CONSIDERING DATA PROVIDED ONE CELL ABOVE TRAIN AN SVM CLASSIFIER
### AND PROVIDE MODEL EVALUATING AND CLASSES / DECISON BOUNDARY VISUALIZATION

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)



### END YOUR BLOCK
from sklearn.datasets import make_circles
from matplotlib import pyplot
from pandas import DataFrame
# generate 2d classification dataset
X, y = make_circles(n_samples=100, noise=0.05)
# scatter plot, dots colored by class value
df = DataFrame(dict(x=X[:,0], y=X[:,1], label=y))
colors = {0:'red', 1:'blue'}
fig, ax = pyplot.subplots()
grouped = df.groupby('label')
for key, group in grouped:
    group.plot(ax=ax, kind='scatter', x='x', y='y', label=key, color=colors[key])
pyplot.show()

### CLASS PRACTICAL TASK 3
### CONSIDERING DATA PROVIDED ONE CELL ABOVE TRAIN AN SVM CLASSIFIER
### AND PROVIDE MODEL EVALUATING AND CLASSES AND DECISON BOUNDARY VISUALIZATION

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)



### AND YOUR BLOCK
### CLASS PRACTICAL TASK 4 - Do here OR in separate nbook
### ASSUMING DATA COME FROM UNKNOWN SOURCE AND HAVE UNKNOWN INTERNAL 
### DISTRIBUTION FORM WE NEED TO DESING A CUSTOM FUNCTION FOR GRID SEARCH
### OVER SVM CLASSIFIERS WITH DIFFERENT KERNELS AND HYPERPARAMETERS.
### FUNCTION MUST HAVE FOLLOWING PROTOTYPE
def find_classifier_grid(X, y):
  
      
        
  
  ### Here you should provide a loop 
  ### That goes throug 5 - 100 predefined options for SVM classifiers
  ### with different kernels and additiona hypperparams
  ### Train each model, evaluate, take the model with best f1 metric
  ### and return the classifier instrance itself as 'clf'
  ### and all SVM hyperparams including kernel inside 'hyperparams' dictionary

  return clf, hyperparams

### END YOUR BLOCK<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11184713375796178, 'fraction_numerical': 0.013885350318471337, 'mean_word_length': 4.301147873058744, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 18, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '87192', 'license': 'MIT', 'n_tokens_mistral': 2600, 'n_tokens_neox': 2412, 'n_words': 882, 'path': 'topics/05_1_SVM_GridSearch_Practice.ipynb', 'repo_name': 'egoliuk/hlll_course', 'size': '7850'}