<jupyter_start><jupyter_text>Working With Support Vector Machines (SVM) for Classification and Detection*<NAME>***Support vector machines (SVMs)** are linear classifiers. An SVM can be understood as a hyperplane (think: line) such that on one side of the plane consists of data only belonging to one class (ideally) while on the other side all instances of the other class exist. Prediction amounts to determining on which side of the line a data point lies.Training an SVM involves finding a hyperplane that best separates data from different classes and trying to maximize the margin between the plane and the nearest data points from two different classes. By doing this SVMs tend to generalize well from training data to future data; they are not known to be prone to overfitting.A hyperparameter common to all SVMs is a parameter $C$ known as the error penalty parameter. Smaller $C$ tends to combat overfitting.SVMs can be trained using the `SVC` object provided in **scikit-learn** Kernel MethodsThe linearity assumption seems restrictive. Analysts overcome it by choosing a **kernel**, a mathematical function that alters the feature space a SVM is trained on. Choosing different kernels allows the boundary between classes to take different shapes, which may lead to better predictions.Again, we can consider choice of kernel as a hyperparameter to optimize. However, we may also use **domain knowledge** (our understanding of the phenomenon being learned) to pick a kernel.Let's load in the *Titanic* dataset again.<jupyter_code>import pandas as pd
from pandas import DataFrame
from sklearn.model_selection import train_test_split, cross_validate
from sklearn.metrics import classification_report
from random import seed
seed(110717)

titanic = pd.read_csv("titanic.csv")
titanic.replace({'Sex': {'male': 0, 'female': 1}}, inplace=True)
titanic.drop("Name", axis=1, inplace=True)
titanic.head()<jupyter_output><empty_output><jupyter_text>Here we would be wise to handle passenger class with more care. While written as numbers this is actually a categorical or ordinal variable; the actual numbers don't matter. We should have binary variables, one for each class.<jupyter_code>pd.get_dummies(titanic.Pclass).head()
titanic = titanic.join(pd.get_dummies(titanic.Pclass, prefix='Pclass')).drop("Pclass", axis=1)
titanic.head()
titanic_train, titanic_test = train_test_split(titanic)
titanic_train.head()<jupyter_output><empty_output><jupyter_text>Training a SVMWe can train a SVM like so:<jupyter_code>from sklearn.svm import SVC
svm1 = SVC(C=1.0,              # Penalty parameter C
           kernel='linear')    # Using a linear kernel
svm1.fit(X=titanic_train.drop("Survived", axis=1), y=titanic_train.Survived)

svm1.predict([[0, 26, 0, 0, 30, 0, 1, 0]])    # Predicting whether a 26 year old male without family aboard in second
                                              # class who paid $30 fare would survive<jupyter_output><empty_output><jupyter_text>Choosing the kernel and $C$ could be done with cross-validation, but I will not demonstrate this (it would take too long for this video).<jupyter_code>print(classification_report(titanic_train.Survived, svm1.predict(titanic_train.drop("Survived", axis=1))))<jupyter_output><empty_output><jupyter_text>The SVM does reasonably well on the training data. Let's see how it does on the test data.<jupyter_code>survived_test_predict = svm1.predict(titanic_test.drop("Survived", axis=1))
print(classification_report(titanic_test.Survived, survived_test_predict))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07675007028394715, 'fraction_numerical': 0.009558616811920157, 'mean_word_length': 5.528440366972477, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 22, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '121739', 'license': 'MIT', 'n_tokens_mistral': 1058, 'n_tokens_neox': 995, 'n_words': 409, 'path': 'Chapter03/SVM.ipynb', 'repo_name': 'PacktPublishing/Training-Systems-using-Python-Statistical-Modeling', 'size': '3564'}