<jupyter_start><jupyter_text>Overfitting In this lab, we are going to explore how thoughtfully choosing a model and using test set are important parts of avoiding overfitting. We are going to apply these concepts to a classification model. We want to develop a decision boundary, on one side of which we have class A and on the other we have class B.Like we disussed in class, when we have noisy data, if we are not careful, we can end up fitting our model to the noise in the data and not the 'signal'-- the factors that actually determine the outcome. This is called overfitting, and results in good results in training, and in bad results when the model is applied to real data. Similarly, we could have a model that is too simplistic to accurately model the signal. This produces a model that doesnt work well (ever), and sucks only slightly less than overfitting. At least your model performs consistently bad :)  First, we are going to generate some synthetic data. To make these concepts visual, we are going to generate this data to be concentric circles. Run the code below to do so.<jupyter_code>import os
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
from IPython.display import display

# Makin' some data
from sklearn.datasets import make_circles
X, y = make_circles(noise=0.2, factor=0.5, random_state=1)<jupyter_output><empty_output><jupyter_text>As our classification algorithm, we are going to use a type of SVM with a radial basis function. This basically works by mapping each point into a higher dimensional space that can be split by the SVM (gross oversimplificaiton). That looks something like this:![RBFSVM.png](../images/RBFSVM.png)  We can change thecomplexity of the decision boundaries applied by the SVM by changignt the size of the radial basis function, through the parameter 'gamma'.Instantiate a list of three SVM classifiers with three different gamma parameters, (.001, 1, and 20).<jupyter_code># Your code here<jupyter_output><empty_output><jupyter_text>Now, we are going to plot the boundaries created by each of these classifiers with the points we generated. Run the following code to make three graphs of each SVM classifier.<jupyter_code>from matplotlib.colors import ListedColormap

figure = plt.figure(figsize=(12, 6))
i = 1
h = .02  # step size in the mesh
x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5
y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5
xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
                     np.arange(y_min, y_max, h))
cm = plt.cm.RdBu
cm_bright = ListedColormap(['#FF0000', '#0000FF'])

# iterate over classifiers
for name, clf in zip(names, classifiers):
    ax = plt.subplot(1, len(classifiers), i)
    clf.fit(X, y)

    # Plot the decision boundary. For that, we will assign a color to each
    # point in the mesh [x_min, x_max]x[y_min, y_max].
    if hasattr(clf, "decision_function"):
        Z = clf.decision_function(np.c_[xx.ravel(), yy.ravel()])
    else:
        Z = clf.predict_proba(np.c_[xx.ravel(), yy.ravel()])[:, 1]

    # Put the result into a color plot
    Z = Z.reshape(xx.shape)
    ax.contourf(xx, yy, Z, cmap=cm, alpha=.8)

    ax.scatter(X[:, 0], X[:, 1], c=y, cmap=cm_bright)

    ax.set_xlim(xx.min(), xx.max())
    ax.set_ylim(yy.min(), yy.max())
    ax.set_xticks(())
    ax.set_yticks(())
    ax.set_title(name)
    i += 1
plt.show()<jupyter_output><empty_output><jupyter_text>Where the contour is red, we will predict red, and same for blue; white means there is a 50/50 chance of either classExplain what you see in the plots above.Which gamma do you think fits the data best? What would you select as the opitmal gamma?<jupyter_code># Your response here<jupyter_output><empty_output><jupyter_text>Without having used a training and testing set, lets see what our accuracy score would be for, for example, a gamma of .7. Use the .score() method.<jupyter_code># Your code here<jupyter_output><empty_output><jupyter_text>Wow thats good accuracy! But is it generalizable? Make a test/train split and see how the model performs on the SVM with the gamma of 20.<jupyter_code># Your code here<jupyter_output><empty_output><jupyter_text>Try with some of the other gammas to see how the accuracy score changes after implementing a test/train split.<jupyter_code># Your code here<jupyter_output><empty_output><jupyter_text>Was your previous model overfitting? If so, how would you try to improve this model to prevent this?<jupyter_code># Your response here<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08773480662983425, 'fraction_numerical': 0.01016574585635359, 'mean_word_length': 4.5060827250608275, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '387221', 'license': 'MIT', 'n_tokens_mistral': 1433, 'n_tokens_neox': 1374, 'n_words': 601, 'path': 'module-3/Problems-in-ML/your-code/lab_overfitting.ipynb', 'repo_name': 'Skye-FY/daft-miami-0120-labs', 'size': '4525'}