<jupyter_start><jupyter_text>Classification using Scikit Learn  Load and prepare the data<jupyter_code># Import library
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
mush_df = pd.read_csv('mushrooms.csv')
mush_df.head()
mush_df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 8124 entries, 0 to 8123
Data columns (total 23 columns):
 #   Column                    Non-Null Count  Dtype 
---  ------                    --------------  ----- 
 0   class                     8124 non-null   object
 1   cap-shape                 8124 non-null   object
 2   cap-surface               8124 non-null   object
 3   cap-color                 8124 non-null   object
 4   bruises                   8124 non-null   object
 5   odor                      8124 non-null   object
 6   gill-attachment           8124 non-null   object
 7   gill-spacing              8124 non-null   object
 8   gill-size                 8124 non-null   object
 9   gill-color                8124 non-null   object
 10  stalk-shape               8124 non-null   object
 11  stalk-root                8124 non-null   object
 12  stalk-surface-above-ring  8124 non-null   object
 13  stalk-surface-below-ring  8124 non-null   object
 14  stalk-color-above-ring  [...]<jupyter_text>Recode categorical variables to numeric  • The data in the mushrooms dataset is **currently encoded with strings**. • These values will **need to be encoded to numeric to work with sklearn**. • We'll use pd.get_dummies to **convert the categorical variables into indicator variables**.<jupyter_code>mush_df2 = pd.get_dummies(mush_df)
display(mush_df2)
mush_df2.info()
# all rows, columns from column at index 2 onwards
# features
X_mush = mush_df2.iloc[:,2:]

# all rows, column at index 1 only
# label
y_mush = mush_df2.iloc[:,1]
display(X_mush)
display(y_mush)
# use the variables X_train2, y_train2 for Question 1
X_train2, X_test2, y_train2, y_test2 =train_test_split(X_mush, y_mush, random_state=0)
# For performance reasons in Questions 6 and 7, we will create a smaller version of the
# entire mushroom dataset for use in those questions. For simplicity we'll just re-use
# the 25% test split created above as the representative subset.

# Use the variables X_subset, y_subset for Questions 2 and 3.
X_subset = X_test2
y_subset = y_test2<jupyter_output><empty_output><jupyter_text>Q1. Using `X_train2` and `y_train2` from the preceding cell, train a DecisionTreeClassifier with default parameters and random_state=0. What are the 5 most important features found by the decision tree? As a reminder, the feature names are available in the `X_train2.columns` property, and the order of the features in `X_train2.columns` matches the order of the feature importance values in the classifier's `feature_importances_` property.   This function should **return a list of length 5 containing the feature names in descending order of importance.**<jupyter_code>def answer_one():
    from sklearn.tree import DecisionTreeClassifier
    
    tree_clf = DecisionTreeClassifier().fit(X_train2, y_train2)
    
    feature_names = []
    
    # Get index of importance leves since their's order is the same with feature columns
    for index, importance in enumerate(tree_clf.feature_importances_):
        # Add importance so we can further order this list, and add feature name with index
        feature_names.append([importance, X_train2.columns[index]])
    
    # Descending sort
    feature_names.sort(reverse=True)
    # Turn in to a numpy array
    feature_names = np.array(feature_names)
    # Select only feature names
    feature_names = feature_names[:5,1]
    # Turn back to python list
    feature_names = feature_names.tolist()
    
    return feature_names # Your answer here
answer_one()<jupyter_output><empty_output><jupyter_text>Q3. Use the `validation_curve` function in `sklearn.model_selection` to determine training and test scores for a Support Vector Classifier (`SVC`) with varying parameter values.   Recall that the validation_curve function, in addition to taking an initialized unfitted classifier object, takes a dataset as input and does its own internal train-test splits to compute results. **Because creating a validation curve requires fitting multiple models, for performance reasons this question will use just a subset of the original mushroom dataset: please use the variables X_subset and y_subset as input to the validation curve function (instead of X_mush and y_mush) to reduce computation time.**The initialized unfitted classifier object we'll be using is a Support Vector Classifier with radial basis kernel.  So your first step is to create an `SVC` object with default parameters (i.e. `kernel='rbf', C=1`) and `random_state=0`. Recall that the kernel width of the RBF kernel is controlled using the `gamma` parameter.   With this classifier, and the dataset in X_subset, y_subset, explore the effect of `gamma` on classifier accuracy by using the `validation_curve` function to find the training and test scores for 6 values of `gamma` from `0.0001` to `10` (i.e. `np.logspace(-4,1,6)`). Recall that you can specify what scoring metric you want validation_curve to use by setting the "scoring" parameter.  In this case, we want to use "accuracy" as the scoring metric.  For each level of `gamma`, `validation_curve` will fit 3 models on different subsets of the data, returning two 6x3 (6 levels of gamma x 3 fits per level) arrays of the scores for the training and test sets. Find the mean score across the three models for each level of `gamma` for both arrays, creating two arrays of length 6, and return a tuple with the two arrays. e.g. if one of your array of scores is array([[ 0.5, 0.4, 0.6],[ 0.7, 0.8, 0.7],[ 0.9, 0.8, 0.8],[ 0.8, 0.7, 0.8],[ 0.7, 0.6, 0.6],[ 0.4, 0.6, 0.5]]) it should then become array([ 0.5, 0.73333333, 0.83333333,0.76666667, 0.63333333, 0.5]) *This function should return one tuple of numpyarrays `(training_scores, test_scores)` where eacharray in the tuple has shape `(6,)`.*<jupyter_code>def answer_two():
    from sklearn.svm import SVC
    from sklearn.model_selection import validation_curve

    svc = SVC(kernel='rbf', C=1, random_state=0)
    gamma = np.logspace(-4,1,6)
    train_scores, test_scores = validation_curve(svc, X_subset, y_subset,
                            param_name='gamma',
                            param_range=gamma,
                            scoring='accuracy')

    scores = (train_scores.mean(axis=1), test_scores.mean(axis=1))
        
    return scores # Your answer here
answer_two()
for index, num in enumerate(np.logspace(-4,1,6)):
    print(num)<jupyter_output>0.0001
0.001
0.01
0.1
1.0
10.0<jupyter_text>Q3. Based on the scores from Q2, what gamma value corresponds to a model that is underfitting (and has the worst test set accuracy)? What gamma value corresponds to a model that is overfitting (and has the worst test set accuracy)? What choice of gamma would be the best choice for a model with good generalization performance on this dataset (high accuracy on both training and test set)? Note: there may be multiple correct solutions to this question.(Hint: Try plotting the scores from question 6 to visualize the relationship between gamma and accuracy.)  *This function should return one tuple with the degree values in this order: `(Underfitting, Overfitting, Good_Generalization)`*<jupyter_code>def answer_three():
    
    param_range = np.logspace(-4, 1, 6)
    
    # Read in the results of answer_six
    training_scores, test_scores = answer_two()
    
    # Sort the scores
    train_scores_sorted = np.sort(training_scores)
    test_scores_sorted = np.sort(test_scores)
    
    # Initialize the values
    Underfitting = 0
    Overfitting = 0
    Good_Generalization = 0
    min_train_scores = np.min(training_scores)
    max_train_scores = np.max(training_scores)
    min_test_scores = np.max(test_scores)
    max_test_scores = np.max(test_scores)    
    
    for gam, data in zip(param_range, zip(training_scores, test_scores)):
        if data[0] <= train_scores_sorted[1] and data[1] <= test_scores_sorted[1]:
            Underfitting = gam
        if data[0] > train_scores_sorted[1] and data[1] <= test_scores_sorted[1]:
            Overfitting = gam
        if data[0] == max_train_scores and data[1] == max_test_scores:
            Good_Generalization = gam
    
    return Underfitting, Overfitting, Good_Generalization
answer_three()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07691405792003751, 'fraction_numerical': 0.03165670066830813, 'mean_word_length': 3.4753410283315844, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 3, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '206629', 'license': 'MIT', 'n_tokens_mistral': 2689, 'n_tokens_neox': 2448, 'n_words': 973, 'path': 'documents/classification_mushroom.ipynb', 'repo_name': 'liawbeile/liawbeile.github.io', 'size': '8529'}