<jupyter_start><jupyter_text>Support Vector Machines with Scikit-learn* SVM offers very high accuracy compared to other classifiers such as logistic regression, and decision trees. * It is known for its kernel trick to handle nonlinear input spaces[when Data is not Separable]. * If your data is Separable then we called it is Linear Spaces.* Given a set of training examples, each marked as belonging to `one or the other of two categories`, an `SVM training algorithm` builds a model that assigns new examples to one category or the other, making it a `non-probabilistic binary linear classifier` (although methods such as Platt scaling exist to use SVM in a probabilistic classification setting).* An SVM model is a representation of the examples as points in space, mapped so that the examples of the separate categories are divided by a clear gap that is as wide as possible. It is used in a variety of `applications` such as 1. Face detection, 2. Intrusion detection, 3. Classification of emails, 4. News articles and web pages, 5. Classification of genes, and 6. Handwriting recognition.SVM is an exciting algorithm and the concepts are relatively simple. The classifier separates data points using a hyperplane with the largest amount of margin. That's why an SVM classifier is also known as a **discriminative classifier**. SVM finds an optimal hyperplane which helps in classifying new data points.**Support Vector Machines*** Generally, Support Vector Machines is considered to be a classification approach, it but can be employed in both types of classification and regression problems. * It can easily handle **multiple continuous(Numerical Data) and categorical variables(Descrete)**. * SVM constructs a `hyperplane in multidimensional space` to separate different classes.* SVM generates optimal hyperplane in an iterative manner, which is used to minimize an error. * `The core idea of SVM is to find a maximum marginal hyperplane(MMH) that best divides the dataset into classes`.or A Support Vector Machine (SVM) is a discriminative classifier formally defined by a separating hyperplane. In other words, given labeled training data (supervised learning), the algorithm outputs an optimal hyperplane which categorizes new examples. In two dimentional space this hyperplane is a line dividing a plane in two parts where in each class lay in either side.![](Images/svm.png)Suppose you are given plot of `two label classes` on graph as shown in image (A). Can you decide a separating line for the classes?![image.png](attachment:image.png) B). It fairly separates the two classes. Any point that is left of line falls into black circle class and on right falls into blue square class. Separation of classes. That’s what SVM does. It finds out a line/ hyper-plane (in multidimensional space that separate outs classes). Shortly, we shall discuss why I wrote multidimensional space.![image.png](attachment:image.png) 1. **Bit Complex**So far so good. Now consider what if we had data as shown in image below? Clearly, there is no line that can separate the two classes in this x-y plane. So what do we do? We apply transformation and add one more dimension as we call it z-axis. Lets assume value of points on z plane, w = x² + y². In this case we can manipulate it as distance of point from z-origin. Now if we plot in z-axis, a clear separation is visible and a line can be drawn .Can you draw a separating line in this plane?![image.png](attachment:image.png) plot of zy axis. A separation can be made here.![image.png](attachment:image.png) When we transform back this line to original plane, it maps to circular boundary as shown in image E. These transformations are called kernels.![image.png](attachment:image.png)Transforming back to x-y plane, a line transforms to circle.![](https://miro.medium.com/max/1200/0*iDZMC0LsciIANcYv.gif)**Basic SVM**<jupyter_code>from sklearn import svm
import matplotlib.pyplot as plt
X = [[0, 0,0], [1, 1,-1]]
y = [0, 1]
clf = svm.SVC(gamma='auto')
clf
clf.fit(X, y)  
clf.predict([[1,0,1]])
# get support vectors
clf.support_vectors_
# get indices of support vectors
clf.support_ 
# get number of support vectors for each class
clf.n_support_<jupyter_output><empty_output><jupyter_text>**SVM: Maximum margin separating hyperplane**<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from sklearn import svm
from sklearn.datasets import make_blobs
# we create 40 separable points
X, y = make_blobs(n_samples=40,centers=2,random_state=6)
X.mean()
X[0:5]
y
# fit the model, don't regularize for illustration purposes
clf = svm.SVC(kernel='linear')
clf.fit(X, y)
clf.decision_function(X)
clf.support_ #
clf.support_vectors_  # +d, -d
clf.coef0
clf.coef_
clf.classes_<jupyter_output><empty_output><jupyter_text>svm.SVC.<jupyter_code>plt.figure(figsize=(20,10))
plt.scatter(X[:, 0], 
            X[:, 1], 
            c=y, 
            s=100, 
            cmap=plt.cm.rainbow)

## plot the decision function
ax = plt.gca()
xlim = ax.get_xlim()
ylim = ax.get_ylim()


# create grid to evaluate model
xx = np.linspace(xlim[0], xlim[1], 30)
yy = np.linspace(ylim[0], ylim[1], 30)
YY, XX = np.meshgrid(yy, xx)
#print(YY)
xy = np.vstack([XX.ravel(),YY.ravel()]).T
#print(xy)
Z = clf.decision_function(xy).reshape(XX.shape)


# plot decision boundary and margins Lines
ax.contour(XX,YY,Z,
          colors='k',
          levels=[-1,0,1],
          alpha=0.5,
          linestyles=['--', '-', '--'])

ax.scatter(clf.support_vectors_[:,0],
           clf.support_vectors_[:,1],
           s=2,
           linewidth=15,
           facecolors='k', 
           edgecolors='g'
          )<jupyter_output><empty_output><jupyter_text>**SVM For IRIS**<jupyter_code>"""
SVM for using different SVM kernels.
"""
import pandas as pd
import matplotlib.pyplot as plt
from sklearn import datasets, svm
iris = datasets.load_iris()
X = iris.data
y = iris.target
X = X[y != 0, :2]
print(len(X))
y = y[y != 0]
print(y)
n_sample = len(X)

np.random.seed(0)
order = np.random.permutation(n_sample)
X = X[order]
y = y[order].astype(np.float)
X_train = X[:int(.9 * n_sample)]
y_train = y[:int(.9 * n_sample)]
X_test = X[int(.9 * n_sample):]
y_test = y[int(.9 * n_sample):]
# fit the model
for kernel in ('linear', 'rbf', 'poly'):
    clf = svm.SVC(kernel=kernel, gamma='auto')
    clf.fit(X_train, y_train)
    print("My Coef is of Each kernal",clf.co)

    plt.figure(figsize=(20,10))
    plt.clf()
    plt.scatter(X[:, 0], X[:, 1], c=y, zorder=10, cmap=plt.cm.Paired,
                edgecolor='k', s=20)
    # Circle out the test data
    plt.scatter(X_test[:, 0], X_test[:, 1], s=80, facecolors='none',zorder=10, edgecolor='k')

    plt.axis('tight')
    x_min = X[:, 0].min()
    x_max = X[:, 0].max()
    y_min = X[:, 1].min()
    y_max = X[:, 1].max()

    XX, YY = np.mgrid[x_min:x_max:200j, y_min:y_max:200j]
    Z = clf.decision_function(np.c_[XX.ravel(), YY.ravel()])
    # Put the result into a color plot
    Z = Z.reshape(XX.shape)
    plt.pcolormesh(XX, YY, Z > 0, cmap=plt.cm.Paired)
    plt.contour(XX, YY, Z, colors=['k', 'k', 'k'],
                linestyles=['--', '-', '--'], levels=[-.5, 0, .5])

    plt.title(kernel)
plt.show()<jupyter_output>c:\users\reddy\appdata\local\programs\python\python36\lib\site-packages\ipykernel_launcher.py:23: MatplotlibDeprecationWarning: shading='flat' when X and Y have the same dimensions as C is deprecated since 3.3.  Either specify the corners of the quadrilaterals with X and Y, or pass shading='auto', 'nearest' or 'gouraud', or set rcParams['pcolor.shading'].  This will become an error two minor releases later.
c:\users\reddy\appdata\local\programs\python\python36\lib\site-packages\ipykernel_launcher.py:23: MatplotlibDeprecationWarning: shading='flat' when X and Y have the same dimensions as C is deprecated since 3.3.  Either specify the corners of the quadrilaterals with X and Y, or pass shading='auto', 'nearest' or 'gouraud', or set rcParams['pcolor.shading'].  This will become an error two minor releases later.
c:\users\reddy\appdata\local\programs\python\python36\lib\site-packages\ipykernel_launcher.py:23: MatplotlibDeprecationWarning: shading='flat' when X and Y have the same dimensio[...]
{'fraction_non_alphanumeric': 0.11031204290589956, 'fraction_numerical': 0.013895660653339834, 'mean_word_length': 4.2663671373555845, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 17, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '378454', 'license': 'MIT', 'n_tokens_mistral': 2718, 'n_tokens_neox': 2558, 'n_words': 981, 'path': 'Supervised Learning/Classifications/Support Vector Machines/Introduction to SVM .ipynb', 'repo_name': 'free-to-learn/Machine-Learning-Concepts', 'size': '8204'}