<jupyter_start><jupyter_text>**Breast Cancer detection using SVM**A Spport Vector Machine (SVM) is a discriminative classifier formally defined by a separating hyperplane. Margin of SVM tries to separate the data with as a large margin as possible. So, the SVM is sometimes also called a **Large margin classifier** and this is actually a consequences of the optimization problem. **Import Libraries**<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Get the DataWe'll use the built in breast cancer dataset from Scikit Learn. We can get with the load function:<jupyter_code>from sklearn.datasets import load_breast_cancer
cancer = load_breast_cancer()<jupyter_output><empty_output><jupyter_text>The data set is presented in a dictionary form:<jupyter_code># Explore the keys of data
cancer.keys()<jupyter_output><empty_output><jupyter_text>**We can grab information and arrays out of this dictionary to set up our data frame and understanding of the features**:<jupyter_code># Print the description of this Dataset
print(cancer['DESCR'])<jupyter_output>.. _breast_cancer_dataset:

Breast cancer wisconsin (diagnostic) dataset
--------------------------------------------

**Data Set Characteristics:**

    :Number of Instances: 569

    :Number of Attributes: 30 numeric, predictive attributes and the class

    :Attribute Information:
        - radius (mean of distances from center to points on the perimeter)
        - texture (standard deviation of gray-scale values)
        - perimeter
        - area
        - smoothness (local variation in radius lengths)
        - compactness (perimeter^2 / area - 1.0)
        - concavity (severity of concave portions of the contour)
        - concave points (number of concave portions of the contour)
        - symmetry
        - fractal dimension ("coastline approximation" - 1)

        The mean, standard error, and "worst" or largest (mean of the three
        worst/largest values) of these features were computed for each image,
        resulting in 30 features.  For instance, field 0 is Mean Radi[...]<jupyter_text>Set up DataFrame<jupyter_code># Now create a feature dataframe
df_feat = pd.DataFrame(cancer['data'], columns=cancer['feature_names'])
df_feat
# information about DataFrame
df_feat.info()
# Describe the DataFrame
df_feat.describe()
cancer.keys()
# Checkout target names
cancer['target_names']
# Now checkout the cancer 'target' column
cancer['target']
len(cancer['target'])
# Now create a target dataframe
df_target = pd.DataFrame(cancer['target'],columns=['Cancer'])
df_target
df_target['Cancer'].value_counts()<jupyter_output><empty_output><jupyter_text>Exploratory Data Analysis (EDA)We'll skip the Data Visualization part due lack of domain knowladge, since there are so many features that are hard to interpret if you don't have domain knowledge of cancer or tumor cells.  Train Test Split<jupyter_code># Import the model for split the data
from sklearn.model_selection import train_test_split
X = df_feat
y = df_target

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=101)
# Check the train feature matrix
# Dimension : (398x30)
X_train.head()
# Check the train target vector
# Dimension : (398x1)
y_train.head()
# Check the test feature matrix
# Dimension : (171x30)
X_test.head()
# Check the test target vector
# Dimension : (171x1)
y_test.head()
##### DIMENSIONS #####

# df_feat = (569x30)
# df_target = (561x1)
# X_train = (398x30)
# y_train = (398x1)
# X_test = (171x30)
# y_test = (171x1)<jupyter_output><empty_output><jupyter_text>Train the Support Vector Classifier<jupyter_code># Import the Support Vector Classifier model
from sklearn.svm import SVC
# Create a SVC object
model = SVC()
# Train our model with our training data (X_train, y_train)
model.fit(X_train, y_train)<jupyter_output>/home/manish/.local/lib/python3.8/site-packages/sklearn/utils/validation.py:73: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  return f(**kwargs)<jupyter_text>Predictions and EvaluationsNow our model has been trained, let's predict using the trained model.<jupyter_code>predictions = model.predict(X_test)
predictions
from sklearn.metrics import classification_report,confusion_matrix
# Print confusion_matrix
print(confusion_matrix(y_test,predictions))
# Print classification report
print(classification_report(y_test,predictions))<jupyter_output>precision    recall  f1-score   support

           0       0.95      0.85      0.90        66
           1       0.91      0.97      0.94       105

    accuracy                           0.92       171
   macro avg       0.93      0.91      0.92       171
weighted avg       0.93      0.92      0.92       171<jupyter_text>We can search for parameters using a GridSearch!  GridsearchFinding the right parameters (like what C or gamma values to use) is a tricky task! But luckily, we can be a little lazy and just try a bunch of combinations and see what works best! This idea of creating a 'grid' of parameters and just trying out all the possible combinations is called a Gridsearch, this method is common enough that Scikit-learn has this functionality built in with GridSearchCV! The CV stands for cross-validation.GridSearchCV takes a dictionary that describes the parameters that should be tried and a model to train. The grid of parameters is defined as a dictionary, where the keys are the parameters and the values are the settings to be tested.* **Large C** value gives low bias and high varience.* **Small C** value gives high bias and low varience.* **Large gama** value gives High bias and low varience.* **Small gama** value gives low bias and High varience.<jupyter_code># Import the model for Gridsearch
from sklearn.model_selection import GridSearchCV<jupyter_output><empty_output><jupyter_text>One of the great things about GridSearchCV is that it is a meta-estimator. It takes an estimator like SVC, and creates a new estimator, that behaves exactly the same - in this case, like a classifier. You should add refit=True and choose verbose to whatever number you want, higher the number, the more verbose (verbose just means the text output describing the process).<jupyter_code># Create a dictionary for grid parameters ('C' and 'gama')
param_grid = {'C': [0.1, 1, 10, 100, 1000], 'gamma': [1, 0.1, 0.01, 0.001, 0.0001], 'kernel': ['rbf']}
param_grid
# Now create a Gridsearch object
# Passing estimator = SVC
# Passing verbose = 3
grid = GridSearchCV(SVC(),param_grid,refit=True,verbose=3)<jupyter_output><empty_output><jupyter_text>What fit does is a bit more involved then usual. First, it runs the same loop with cross-validation, to find the best parameter combination. Once it has the best combination, it runs fit again on all data passed to fit (without cross-validation), to built a single new model using the best parameter setting.<jupyter_code># Change the shape of y_train to an array
y_train = np.ravel(y_train)
# Now train our model with our training set (X_train, y_train)
grid.fit(X_train,y_train)<jupyter_output>Fitting 5 folds for each of 25 candidates, totalling 125 fits
[CV] C=0.1, gamma=1, kernel=rbf ......................................
[CV] .......... C=0.1, gamma=1, kernel=rbf, score=0.637, total=   0.0s
[CV] C=0.1, gamma=1, kernel=rbf ......................................
[CV] .......... C=0.1, gamma=1, kernel=rbf, score=0.637, total=   0.0s
[CV] C=0.1, gamma=1, kernel=rbf ......................................
[CV] .......... C=0.1, gamma=1, kernel=rbf, score=0.625, total=   0.0s
[CV] C=0.1, gamma=1, kernel=rbf ......................................
[CV] .......... C=0.1, gamma=1, kernel=rbf, score=0.633, total=   0.0s
[CV] C=0.1, gamma=1, kernel=rbf ......................................
[CV] .......... C=0.1, gamma=1, kernel=rbf, score=0.633, total=   0.0s
[CV] C=0.1, gamma=0.1, kernel=rbf ....................................
[CV] ........ C=0.1, gamma=0.1, kernel=rbf, score=0.637, total=   0.0s
[CV] C=0.1, gamma=0.1, kernel=rbf ....................................
[CV] ........ C[...]<jupyter_text>You can inspect the best parameters found by GridSearchCV in the best_params_ attribute, and the best estimator in the best\_estimator_ attribute:<jupyter_code># Check the best parameters combination
grid.best_params_
# Check the best estimator
grid.best_estimator_<jupyter_output><empty_output><jupyter_text>Now we can re-run predictions on this grid object just like we would with a normal model.<jupyter_code># Make predictions with our test data (X_test)
grid_predictions = grid.predict(X_test)
grid_predictions
# Print confusion_matrix
print(confusion_matrix(y_test,grid_predictions))
# Print classification report
print(classification_report(y_test, grid_predictions))<jupyter_output>precision    recall  f1-score   support

           0       0.94      0.89      0.91        66
           1       0.94      0.96      0.95       105

    accuracy                           0.94       171
   macro avg       0.94      0.93      0.93       171
weighted avg       0.94      0.94      0.94       171
{'fraction_non_alphanumeric': 0.12271991365353481, 'fraction_numerical': 0.03151645979492715, 'mean_word_length': 3.4313725490196076, 'pattern_counts': {'":': 0, '<': 48, '<?xml version=': 0, '>': 48, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '53629', 'license': 'MIT', 'n_tokens_mistral': 2927, 'n_tokens_neox': 2720, 'n_words': 1021, 'path': 'MyProjects/13. Breast Cancer detection using ML/SVM.ipynb', 'repo_name': 'Manish-Sharma-1810/Data-Science-Projects', 'size': '9265'}