<jupyter_start><jupyter_text>Support Vector Machines Import Libraries<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Get the DataUsing the built-in breast-cancer dataset of sklearn<jupyter_code>from sklearn.datasets import load_breast_cancer
cancer = load_breast_cancer()<jupyter_output><empty_output><jupyter_text>The data set is presented in a dictionary form:<jupyter_code>cancer.keys()<jupyter_output><empty_output><jupyter_text>We can grab information and arrays out of this dictionary to set up our data frame and understanding of the features:<jupyter_code>print(cancer['DESCR'])
cancer['feature_names']<jupyter_output><empty_output><jupyter_text>Set up DataFrame<jupyter_code>df_feat = pd.DataFrame(cancer['data'],columns=cancer['feature_names'])
df_feat.info()
cancer['target']
df_target = pd.DataFrame(cancer['target'],columns=['Cancer'])
df_target.head()<jupyter_output><empty_output><jupyter_text>Train Test Split<jupyter_code>from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(df_feat, np.ravel(df_target), test_size=0.30, random_state=101)<jupyter_output><empty_output><jupyter_text>Train the Support Vector Classifier<jupyter_code>from sklearn.svm import SVC
model = SVC()
model.fit(X_train,y_train)<jupyter_output><empty_output><jupyter_text>Predictions and Evaluations<jupyter_code>predictions = model.predict(X_test)
from sklearn.metrics import classification_report,confusion_matrix
print(confusion_matrix(y_test,predictions))
print(classification_report(y_test,predictions))<jupyter_output>precision    recall  f1-score   support

          0       0.00      0.00      0.00        66
          1       0.61      1.00      0.76       105

avg / total       0.38      0.61      0.47       171<jupyter_text>Notice that we are classifying everything into a single class! This means our model needs to have it parameters adjusted (it may also help to normalize the data).  GridsearchFinding the right parameters (like what C or gamma values to use) is a tricky task! This idea of creating a 'grid' of parameters and just trying out all the possible combinations is called a Gridsearch, this method is common enough that Scikit-learn has this functionality built in with GridSearchCV! The CV stands for cross-validation which is the GridSearchCV takes a dictionary that describes the parameters that should be tried and a model to train. The grid of parameters is defined as a dictionary, where the keys are the parameters and the values are the settings to be tested.<jupyter_code>param_grid = {'C': [0.1,1, 10, 100, 1000], 'gamma': [1,0.1,0.01,0.001,0.0001], 'kernel': ['rbf']} 
from sklearn.model_selection import GridSearchCV
grid = GridSearchCV(SVC(),param_grid,refit=True,verbose=3)<jupyter_output><empty_output><jupyter_text>What fit does is a bit more involved then usual. First, it runs the same loop with cross-validation, to find the best parameter combination. Once it has the best combination, it runs fit again on all data passed to fit (without cross-validation), to built a single new model using the best parameter setting.<jupyter_code># May take awhile!
grid.fit(X_train,y_train)<jupyter_output>Fitting 3 folds for each of 25 candidates, totalling 75 fits
[CV] kernel=rbf, C=0.1, gamma=1 ......................................
[CV] ....... kernel=rbf, C=0.1, gamma=1, score=0.631579, total=   0.0s
[CV] kernel=rbf, C=0.1, gamma=1 ......................................
[CV] ....... kernel=rbf, C=0.1, gamma=1, score=0.631579, total=   0.0s
[CV] kernel=rbf, C=0.1, gamma=1 ......................................
[CV] ....... kernel=rbf, C=0.1, gamma=1, score=0.636364, total=   0.0s
[CV] kernel=rbf, C=0.1, gamma=0.1 ....................................
[CV] ..... kernel=rbf, C=0.1, gamma=0.1, score=0.631579, total=   0.0s
[CV] kernel=rbf, C=0.1, gamma=0.1 ....................................
[CV] ..... kernel=rbf, C=0.1, gamma=0.1, score=0.631579, total=   0.0s
[CV] kernel=rbf, C=0.1, gamma=0.1 ....................................
[CV] ..... kernel=rbf, C=0.1, gamma=0.1, score=0.636364, total=   0.0s
[CV] kernel=rbf, C=0.1, gamma=0.01 ...................................
[CV] .... kernel[...]<jupyter_text>You can inspect the best parameters found by GridSearchCV in the best_params_ attribute, and the best estimator in the best\_estimator_ attribute:<jupyter_code>grid.best_params_
grid.best_estimator_<jupyter_output><empty_output><jupyter_text>Then you can re-run predictions on this grid object just like you would with a normal model.<jupyter_code>grid_predictions = grid.predict(X_test)
print(confusion_matrix(y_test,grid_predictions))
print(classification_report(y_test,grid_predictions))<jupyter_output>precision    recall  f1-score   support

          0       0.95      0.91      0.93        66
          1       0.94      0.97      0.96       105

avg / total       0.95      0.95      0.95       171
{'fraction_non_alphanumeric': 0.15254574383452665, 'fraction_numerical': 0.04295942720763723, 'mean_word_length': 3.6435826408125576, 'pattern_counts': {'":': 0, '<': 46, '<?xml version=': 0, '>': 46, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '293925', 'license': 'MIT', 'n_tokens_mistral': 1737, 'n_tokens_neox': 1615, 'n_words': 461, 'path': 'Support Vector Machines/SVM.ipynb', 'repo_name': 'Shubh98/Lectures-On-Machine-Learning', 'size': '5028'}